metrics

My colleagues Dinei Florencio and Cormac Herley have a new paper out, “Sex, Lies and Cyber-crime Surveys.” Our assessment of the quality of cyber-crime surveys is harsh: they are so compromised and biased that no faith whatever can be placed in their findings. We are not alone in this judgement. Most research teams who have…

Read More Sex, Lies & Cybercrime Surveys: Getting to Action

In two recent blog posts (here and here), Chris Wysopal (CTO of Veracode) proposed a metric called “Application Security Debt”.  I like the general idea, but I have found some problems in his method.  In this post, I suggest corrections that will be both more credible and more accurate, at least for half of the…

Read More Fixes to Wysopal’s Application Security Debt Metric

Mike Rothman’s “Firestarter” on “Risk Metrics are Crap“. It’s very difficult to argue with a poorly constructed argument.  Especially when I have no idea what a “risk metric” is.  But best as I can tell, Mike’s position is that unless you are smart and/or have strong resources allocated to your InfoSec team, things like metrics,…

Read More Just Because YOU Think Your Clients Are Too Busy and/or Stupid Doesn't Mean Everyone Else Is

Symantec’s new Norton Cybercrime Index looks like it is mostly a marketing tool. They present it as though there is solid science, data, and methods behind it, but an initial analysis shows that this is probably not the case. The only way to have confidence in this is if Symantec opens up about their algorthms and data.

Read More Is Norton Cybercrime Index just 'Security Metrics Theater'?

The visual metaphor of a dashboard is a dumb idea for management-oriented information security metrics. It doesn’t fit the use cases and therefore doesn’t support effective user action based on the information. Dashboards work when the user has proportional controllers or switches that correspond to each of the ‘meters’ and the user can observe the effect of using those controllers and switches in real time by observing the ‘meters’. Dashboards don’t work when there is a loose or ambiguous connection between the information conveyed in the ‘meters’ and the actions that users might take. Other visual metaphors should work better.

Read More Dashboards are Dumb

No doubt my “Why I Don’t Like CRISC” blog post has created a ton of traffic and comments.  Unfortunately, I’m not a very good writer because the majority of readers miss the point.  Let me try again more succinctly: Just because you can codify a standard or practice doesn’t mean that this practice is sane.…

Read More CRISC – The Bottom Line (oh yeah, Happy New Year!)