Shostack + Friends Blog Archive

 

Survey for How to Measure Anything In Cybersecurity Risk

This is a survey from Doug Hubbard, author of How To Measure Anything and he is currently writing another book with Richard Seiersen (GM of Cyber Security at GE Healthcare) titled How to Measure Anything in Cybersecurity Risk. As part of the research for this book, they are asking for your assistance as an information […]

 

A Mini-Review of "The Practice of Network Security Monitoring"

Recently the kind folks at No Starch Press sent me a review copy of Rich Bejtlich’s newest book The Practice of Network Security Monitoring and I can’t recommend it enough. It is well worth reading from a theory perspective, but where it really shines is digging into the nuts and bolts of building an NSM […]

 

Active Defense: Show me the Money!

Over the last few days, there’s been a lot of folks in my twitter feed talking about “active defense.” Since I can’t compress this into 140 characters, I wanted to comment quickly: show me the money. And if you can’t show me the money, show me the data. First, I’m unsure what’s actually meant by […]

 

Oracle's 78 Patches This Quarter, Whatever…

There’s been a lot of noise of late because Oracle just released their latest round of patches and there are a total of 78 of them. There’s no doubt that that is a lot of patches. But in and of itself the number of patches is a terrible metric for how secure a product is. […]

 

Seattle in the Snow

(From The Oatmeal.) It’s widely understood that Seattle needs a better way to measure snowfall. However, what’s lacking is a solid proposal for how to measure snowfall around here. And so I have a proposal. We should create a new unit of measurement: The Nickels. Named after Greg Nickels, who lost the mayorship of Seattle […]

 

Lean Startups & the New School

On Friday, I watched Eric Ries talk about his new Lean Startup book, and wanted to talk about how it might relate to security. Ries concieves as startups as businesses operating under conditions of high uncertainty, which includes things you might not think of as startups. In fact, he thinks that startups are everywhere, even […]

 

Securosis goes New School

The fine folks at Securosis are starting a blog series on “Fact-based Network Security: Metrics and the Pursuit of Prioritization“, starting in a couple of weeks.  Sounds pretty New School to me!  I suggest that you all check it out and participate in the dialog.  Should be interesting and thought provoking. [Edit — fixed my […]

 

Emergent Map: Streets of the US

This is really cool. All Streets is a map of the United States made of nothing but roads. A surprisingly accurate map of the country emerges from the chaos of our roads: All Streets consists of 240 million individual road segments. No other features — no outlines, cities, or types of terrain — are marked, […]

 

Sex, Lies & Cybercrime Surveys: Getting to Action

My colleagues Dinei Florencio and Cormac Herley have a new paper out, “Sex, Lies and Cyber-crime Surveys.” Our assessment of the quality of cyber-crime surveys is harsh: they are so compromised and biased that no faith whatever can be placed in their findings. We are not alone in this judgement. Most research teams who have […]

 
 

Fixes to Wysopal’s Application Security Debt Metric

In two recent blog posts (here and here), Chris Wysopal (CTO of Veracode) proposed a metric called “Application Security Debt”.  I like the general idea, but I have found some problems in his method.  In this post, I suggest corrections that will be both more credible and more accurate, at least for half of the […]

 
 

Is Norton Cybercrime Index just 'Security Metrics Theater'?

Symantec’s new Norton Cybercrime Index looks like it is mostly a marketing tool. They present it as though there is solid science, data, and methods behind it, but an initial analysis shows that this is probably not the case. The only way to have confidence in this is if Symantec opens up about their algorthms and data.

 

Gunnar on Heartland

Analysis of Heartland’s business as a going concern by @oneraindrop. Especially interesting after comments on the CMO video.

 

Dashboards are Dumb

The visual metaphor of a dashboard is a dumb idea for management-oriented information security metrics. It doesn’t fit the use cases and therefore doesn’t support effective user action based on the information. Dashboards work when the user has proportional controllers or switches that correspond to each of the ‘meters’ and the user can observe the effect of using those controllers and switches in real time by observing the ‘meters’. Dashboards don’t work when there is a loose or ambiguous connection between the information conveyed in the ‘meters’ and the actions that users might take. Other visual metaphors should work better.

 

CRISC – The Bottom Line (oh yeah, Happy New Year!)

No doubt my “Why I Don’t Like CRISC” blog post has created a ton of traffic and comments.  Unfortunately, I’m not a very good writer because the majority of readers miss the point.  Let me try again more succinctly: Just because you can codify a standard or practice doesn’t mean that this practice is sane. […]

 

The Only Trust Models You'll Ever Need

Lately there has been quite a bit of noise about the concept of “trust” in information security.  This has always confused me, because I tend towards @bobblakley when he says: “trust is for suckers.” But security is keen on having trendy new memes, things to sell you, and I thought that I might as well […]

 

Visualization for Gunnar's "Heartland Revisited"

You may have heard me say in the past that one of the more interesting aspects of security breaches, for me at least, is the concept of reputation damage.  Maybe that’s because I heard so many sales tactics tied to defacement in the 90’s, maybe because it’s so hard to actually quantify brand equity and […]

 

A Letter from Sid CRISC – ious

In the comments to “Why I Don’t Like CRISC” where I challenge ISACA to show us in valid scale and in publicly available models, the risk reduction of COBIT adoption, reader Sid starts to get it, but then kinda devolves into a defense of COBIT or something.  But it’s a great comment, and I wanted […]

 

Fines or Reporting?

Over at the Office of Inadequate Security, Dissent does excellent work digging into several perspectives on Discover Card breaches: Discover’s reports, and the (apparent) silence of breached entities. I’m concerned that for many of the breaches they report, we have never seen breach reports filed by the entities themselves nor media reports on the incidents. […]

 

What They Know (From the WSJ)

Interesting interactive data app from the Wall Street Journal about your privacy online and what various websites track/know about you. http://blogs.wsj.com/wtk/ Full disclosure, our site uses Mint for traffic analytics.

 

Measuring The Speed of Light Using Your Microwave

Using a dish full of marshmallows.  We’re doing this with my oldest kids, and while I was reading up on it, I had to laugh out loud at the following: …now you have what you need to measure the speed of light. You just need to know a very fundamental equation of physics: Speed of […]

 

Getting the time dimension right

If you are developing or using security metrics, it’s inevitable that you’ll have to deal with the dimension of time. “Data” tells you about the past. “Security” is a judgement about the present. “Risk” is a cost of the future, brought to the present. The way to marry these three is through social learning processes.

 

On Uncertain Security

One of the reasons I like climate studies is because the world of the climate scientist is not dissimilar to ours.  Their data is frought with uncertainty, it has gaps, and it might be kind of important (regardless of your stance of anthropomorphic global warming, I think we can all agree that when the climate […]

 

Your credit worthiness in 140 Characters or Less

In “Social networking: Your key to easy credit?,” Eric Sandberg writes: In their quest to identify creditworthy customers, some are tapping into the information you and your friends reveal in the virtual stratosphere. Before calling the privacy police, though, understand how it’s really being used. … To be clear, creditors aren’t accessing the credit reports […]

 

Data void: False Positives

A Gartner blog post points out the lack of data reported by vendors or customers regarding the false positive rates for anti-spam solutions. This is part of a general problem in the security industry that is a major obstical to rational analysis of effectiveness, cost-effectiveness, risk, and the rest

 

Does It Matter If The APT Is "New"?

As best as I can describe the characteristics of the threat agents that would fit the label of APT, that threat community is very, very real.  It’s been around forever (someone mentioned first use of the term being 1993 or something) – we dealt with threat agents you would describe as “APT” at MicroSovled when […]

 

V-22 Osprey Metrics

Metrics seem to be yet another way in which Angry Bear noticed that the V-22 Osprey program has hidden from its failure to deliver on its promises: Generally, mission capability runs 20% higher than availability, but availability is hidden on new stuff, while shouted about on older stuff, because there would be severe embarrassment if you […]

 

Help EFF Measure Browser Uniqueness

The EFF is doing some measurement of browser uniqueness and privacy. It takes ten seconds. Before you go, why not estimate what fraction of users have the same transmitted/discoverable browser settings as you, and then check your accuracy at https://panopticlick.eff.org. Or start at http://www.eff.org/deeplinks/2010/01/help-eff-research-web-browser-tracking for a bit more detail.

 

NotObvious On Heartland

I posted this also to the securitymetrics.org mailing list.  Sorry if discussing in multiple  venues ticks you off. The Not Obvious blog has an interesting write up on the Heartland Breach and impact.  From the blog post: “Heartland has had to pay other fines to Visa and MasterCard, but the total of $12.6 million they […]

 

Sweden: An Interesting Demographic Case Study In Internet Fraud

(quietly, wistfully singing “Yesterday” by the Beatles) From my favorite Swedish Infosec Blog, Crowmoor.se. I don’t speak Swedish, so I couldn’t really read the fine article they linked to.  Do go read their blog post, I’ll wait here. Back?  Great.  Here are my thoughts on those numbers: SWEDISH FRAUD STATISTICS RELEASED The World Bank estimates […]

 

Chris Soghoian’s Surveillance Metrics

I also posted about this on Emergent Chaos, but since our readership doesn’t fully overlap, I’m commenting on it here as well. Chis Soghoian, has just posted some of his new research into government electronic surveillance here in the US. The numbers are truly astounding (Sprint for instance provided geo-location data on customers eight million […]

 

For Those Not In The US (or even if you are)

I’d like to wish US readers a happy Thanksgiving. For those outside of the US, I thought this would be a nice little post for today: A pointer to an article in the Financial Times, “Baseball’s love of statistics is taking over football“ Those who indulge my passion for analysis and for sport know that […]

 

Rational Ignorance: The Users' view of security

Cormac Herley at Microsoft Research has done us all a favor and released a paper So Long, And No Thanks for the Externalities:  The Rational Rejection of Security Advice by Users which opens its abstract with: It is often suggested that users are hopelessly lazy and unmotivated on security questions. They chose weak passwords, ignore […]

 

Metrics: 50% Chance of Injury by Biscuit

The Telegraph reports: More than half of all Britons have been injured by biscuits ranging from scalding from hot tea or coffee while dunking or breaking a tooth eating during a morning tea break, a survey has revealed. Who knew that cookies could be so dangerous? So forget worrying about AV or even seat belts, […]

 

2008 Breaches: More or More Reporting?

Dissent has some good coverage of an announcement from the ID Theft Resource Center, “ITRC: Breaches Blast ’07 Record:” With slightly more than four months left to go for 2008, the Identity Theft Resource Center (ITRC) has sent out a press release saying that it has already compiled 449 breaches– more than its total for […]

 

Laptops and border crossings

The New York Times has in an editorial, “The Government and Your Laptop” a plea for Congress to pass a law to ensure that laptops (along with phones, etc.) are not seized at borders without reasonable suspicion. The have the interesting statistic that in a survey by the Association of Corporate Travel Executives, 7 of […]

 

Department of Justice on breach notice

There’s an important new report out from the Department of Justice, “Data Breaches: What the Underground World of “Carding” Reveals.” It’s an analysis of several cases and the trends in carding and the markets which exist. I want to focus in on one area, which is recommendations around breach notification: Several bills now before Congress […]

 

Security Prediction Markets: theory & practice

There are a lot of great comments on the “Security Prediction Markets” post. There’s a tremendous amount of theorizing going on here, and no one has any data. Why don’t we experiment and get some? What would it take to create a market in breach notification prediction? Dan Guido said in a comment, “In security, […]

 

Security Prediction Markets?

In our first open thread, Michael Cloppert asked: Considering the contributors to this blog often discuss security in terms of economics, I’m curious what you (and any readers educated on the topic) think about the utility of using prediction markets to forecast compromises. So I’m generally a big fan of markets. I think markets are, […]

 

Why the heck don't I ever have ideas this good?

Walkscore.com. Calculates a location’s “walkability” by using Google Maps to figure out how close various amenities (such as grocery stores, public transit, parks, etc.) are. Not a perfect service, but a great idea.

 

Quantum Progress

What is it about the word “quantum” that sucks the brains out of otherwise reasonable people? There has to be some sort of Heisenberg-Schödinger Credulity Principle that makes all the ideons in their brains go spin-up at the same time, and I’m quite sure that the Many Worlds Interpretation of it has the most merit. […]

 

Are We Measuring the Right Things?

One of the reasons that airline passengers sit on the tarmac for hours before takeoff is how the FAA Department of Transportation measures “on time departures.” The on time departure is measured by push-back from the gate, not wheels leaving the tarmac. (Airlines argue that the former is in their control.) If you measure the […]

 
 

Measuring the Wrong Stuff

There’s a great deal of discussion out there about security metrics. There’s a belief that better measurement will improve things. And while I don’t disagree, there are substantial risks from measuring the wrong things: Because the grades are based largely on improvement, not simply meeting state standards, some high-performing schools received low grades. The Clove […]

 

Defending Metrics

Yesterday, I attacked metrics claiming that the way they are being used today, they were useless to upper management and didn’t relate the value of the InfoSec team to the business. While I stand behind that claim, also believe that a lot of metrics being performed today are very useful to technical management especially those […]

 

Attacking Metrics

Last week I had the pleasure of having lunch with Alex Hutton from RMI and we got to talking about metrics. Specifically, we talked about how most metrics that we security folks come up with are well boring are effectively useless to upper management. At best they are focused on technical management such as the […]