Category: Science of Risk Management

The NTSB has released “Preliminary Report Highway HWY18MH010,” on the Uber self-driving car which struck and killed a woman. I haven’t had a chance to read the report carefully.

Brad Templeton has excellent analysis of the report at “NTSB Report implies serious fault for Uber in fatality” (and Brad’s writings overall on the subject have been phenomenal.)

A few important things to note, cribbed from Brad.

  • The driver was not looking at her phone, but a screen with diagnostic information from the self-driving systems.
  • The car detected a need to brake with approximately enough time to stop had it automatically applied the brakes.
  • That system was turned off for a variety of reasons that look bad (in hindsight, and probably could have been critiqued at the time).

My only comment right now is wouldn’t it be nice to have this level of fact finding in the world of cyber?

Also, it’s very clear that the vehicle was carefully preserved. Can anyone say how the NTSB and/or Uber preserved the data center, cloud or other remote parts of the computer systems involved, including the algorithms that were deployed that day (versus reconstructing them later)?

What’s more primordial than fire? It’s easy to think that fire is a static threat, and defenses against it can be static. So it was surprising to see that changes in home design and contents are leading to fires spread much faster, and that the Canadian Commission on Building and Fire Codes is considering mandates for home sprinklers.

The CBC’s “Rise in fast-burning house fires heats up calls for sprinklers in homes” has a good discussion of the changing threat, the costs of mitigation, and the tradeoffs entailed.

Bicycling & Risk

While everyone else is talking about APT, I want to talk about risk thinking versus outcome thinking.

I have a lot of colleagues who I respect who like to think about risk in some fascinating ways. For example, there’s the Risk Hose and SIRA folks.
I’m inspired by
To Encourage Biking, Cities Lose the Helmets:

In the United States the notion that bike helmets promote health and safety by preventing head injuries is taken as pretty near God’s truth. Un-helmeted cyclists are regarded as irresponsible, like people who smoke. Cities are aggressive in helmet promotion.

But many European health experts have taken a very different view: Yes, there are studies that show that if you fall off a bicycle at a certain speed and hit your head, a helmet can reduce your risk of serious head injury. But such falls off bikes are rare — exceedingly so in mature urban cycling systems.

On the other hand, many researchers say, if you force or pressure people to wear helmets, you discourage them from riding bicycles. That means more obesity, heart disease and diabetes. And — Catch-22 — a result is fewer ordinary cyclists on the road, which makes it harder to develop a safe bicycling network. The safest biking cities are places like Amsterdam and Copenhagen, where middle-aged commuters are mainstay riders and the fraction of adults in helmets is minuscule.

“Pushing helmets really kills cycling and bike-sharing in particular because it promotes a sense of danger that just isn’t justified.

Given that we don’t have statistics about infosec analogs to head injuries, nor obesity, I’m curious where can we make the best infosec analogy to bicycling and helmets? Where are our outcomes potentially worse because we focus on every little risk?

My favorite example is password change policies, where we absorb substantial amounts of everyone’s time without evidence that they’ll improve our outcomes.

What’s yours?

New paper: "How Bad Is It? — A Branching Activity Model for Breach Impact Estimation"

Adam just posted a question about CEO “willingness to pay” (WTP) to avoid bad publicity regarding a breach event.  As it happens, we just submitted a paper to Workshop on the Economics of Information Security (WEIS) that proposes a breach impact estimation method that might apply to Adam’s question.  We use the WTP approach in a specific way, by posing this question to all affected stakeholders:

Ex ante, how much would you be willing to spend on response and recovery for a breach of a particular type?  Through what specific activities and processes?”

We hope this approach can bridge theoretical and empirical research, and also professional practice.  We also hope that this method can be used in public disclosures.

Paper: How Bad is it? – A Branching Activity Model to Estimate the Impact of Information Security Breaches

Infographic from the example in the paper
Infographic from the example in the paper

In the next few months we will be applying this to half a dozen historical breach episodes to see how it works out.  This model will also probably find its way into my dissertation as “substrate”.  The dissertation focus is on social learning and institutional innovation.

Comments and feedback are most welcome.

Base Rate & Infosec

At SOURCE Seattle, I had the pleasure of seeing Jeff Lowder and Patrick Florer present on “The Base Rate Fallacy.” The talk was excellent, lining up the idea of the base rate fallacy, how and why it matters to infosec. What really struck me about this talk was that about a week before, I had read a presentation of the fallacy with exactly the same example in Kahneman’s “Thinking, Fast and Slow.” The problem is you have a witness who’s 80% accurate, describing a taxi as orange; what are the odds she’s right, given certain facts about the distribution of taxis in the city?

I had just read the discussion. I recognized the problem. I recognized that the numbers were the same. I recalled the answer. I couldn’t remember how to derive it, and got the damn thing wrong.

Well played, sirs! Game to Jeff and Patrick.

Beyond that, there’s an important general lesson in the talk. It’s easy to make mistakes. Even experts, primed for the problems, fall into traps and make mistakes. If we publish only our analysis (or worse, engage in information sharing), then others can’t see what mistakes we might have made along the way.

This problem is exacerbated in a great deal of work by a lack of a methodology section, or a lack of clear definitions.

The more we publish, the more people can catch one anothers errors, and the more the field can advance.

Aitel on Social Engineering

Yesterday, Dave Aitel wrote a fascinating article “Why you shouldn’t train employees for security awareness,” arguing that money spent on training employees about awareness is wasted.

While I don’t agree with everything he wrote, I submit that your opinion on this (and mine) are irrelevant. The key question is “Is money spent on security awareness a good way to spend your security budget?”

The key is we have data now. As someone comments:

[Y]ou somewhat missed the point of the phishing awareness program. I do agree that annual training is good at communicating policy – but horrible at raising a persistent level of awareness. Over the last 5 years, our studies have shown that annual IA training does very little to improve the awareness of users against social engineering based attacks (you got that one right). However we have shown a significant improvement in awareness as a result of running the phishing exercises (Carronade). These exercises sent fake phishing emails to our cadet population every few weeks (depends on the study). We demonstrated a very consistent reduction to an under 5% “failure” after two iterations. This impact is short lived though. After a couple months of not running the study, the results creep back up to 40% failure.

So, is a reduction in phishing failure to under 5% a good investment? Are there other investments that bring your failure rates lower?

As I pointed out in “The Evolution of Information Security” (context), we can get lots more data with almost zero investment.

If someone (say, GAO) obtains data on US Government department training programs, and cross-correlates that with incidents being reported to US-CERT, then we can assess the efficacy of those training programs

Opinions, including mine, Dave’s, and yours, just ain’t relevant in the face of data. We can answer well-framed questions of “which of these programs best improves security” and “is that improvement superior to other investments?”

The truth, in this instance, won’t set us free. Dave Mortman pointed out that a number of regulations may require awareness training as a best practice. But if we get data, we can, if needed, address the regulations.

If you’re outraged by Dave’s claims, prove him wrong. If you’re outraged by the need to spend money on social engineering, prove it’s a waste.

Put the energy to better use than flaming over a hypothesis.

Yet More On Threat Modeling: A Mini-Rant

Yesterday Adam responded to Alex’s question on what people thought about IanG’s claim that threat modeling fails in practice and I wanted to reiterate what I said on twitter about it:

It’s a tool! No one claimed it was a silver bullet!

Threat modeling is yet another input into an over all risk analysis. And you know what? Risk analysis/Risk management, whatever you want to call it won’t be perfect either. Threat modeling is in itself a model. All models are broken. We’ll get better at it.

But claiming that something is a failure because it’s not perfect and that it doesn’t always work, is one of the cardinal sins of infosec from my perspective. Every time we do that, we do ourselves and our industry a disservice. Stop letting the perfect be the enemy of the useful.

Aviation Safety

The past 10 years have been the best in the country’s aviation history with 153 fatalities. That’s two deaths for every 100 million passengers on commercial flights, according to an Associated Press analysis of government accident data.

The improvement is remarkable. Just a decade earlier, at the time the safest, passengers were 10 times as likely to die when flying on an American plane. The risk of death was even greater during the start of the jet age, with 1,696 people dying — 133 out of every 100 million passengers — from 1962 to 1971. The figures exclude acts of terrorism.

There are a number of reasons for the improvements.

  • The industry has learned from the past. New planes and engines are designed with prior mistakes in mind. Investigations of accidents have led to changes in procedures to ensure the same missteps don’t occur again.
  • Better sharing of information. New databases allow pilots, airlines, plane manufactures and regulators to track incidents and near misses. Computers pick up subtle trends. For instance, a particular runway might have a higher rate of aborted landings when there is fog. Regulators noticing this could improve lighting and add more time between landings.

(“It’s never been safer to fly; deaths at record low“, AP, link to Seattle PI version.)

Well, it seems there’s nothing for information security to learn here. Move along.

Discussing Norm Marks' GRC Wishlist for 2012

Norm Marks of the famous Marks On Governance blog has posted his 2012 wishlist.  His blog limits the characters you can leave in a reply, so I thought I’d post mine here.

1.  Norm Wishes for “A globally-accepted organizational governance code, encompassing both risk management and internal control”

Norm, if you mean encompassing both so that they are tightly coupled, I respectfully disagree.  Ethically, philosophically, these should be separate entities and ne’r the twain should meet.  Plus, accountants & auditors make poor actuaries.  See the SoA condemnation of RCSA.

Second, the problem with a globally accepted something is that it limits innovation.  We already have enough of this “we can’t do things right because we’ll have to justify doing things differently than the global priesthood says we have to” problem to deal with now.  Such documentation will only exacerbate the issue.

2.  Norm wishes for: “The convergence of the COSO ERM Framework and the global ISO 31000:2009 risk management standard.”

See #1, part 2 above.

3.  Norm wishes for:  “An update of the COSO Internal Control Framework that recognizes that internal controls are the organization’s response to uncertainty (i.e., risk), and you need the controls to ensure the likelihood and effects of uncertainty are within organizational tolerances.”

First, risk only equals uncertainty if you’re one of those stuck in the early 20th century Knightians.  For those that aren’t, and esp. actuaries and Bayesians alike, uncertainty is a factor in risk analysis – not the existence of risk.

Second, this wish seems to be beholden to the fundamental flaw of the Accounting Consultancy Industrial Complex – that Residual Risk = Inherent risk – Controls.  Let me ask you, what controls do you personally have against an asteroid slamming into your house?  But is that “high” risk?  Do you operate daily as if it’s “high” risk?  Why not?  Certainly you have weak controls, and most people would argue that their house and familys are of high value…

The reason it’s not “high risk” is because of frequency.  Yes, frequency matters in risk – and your RCSA process doesn’t (usually, formally) account for that.

4.) Norm wants “guidance that explains how you can set guidance on risk-taking that works not only for (a) the board and top management (who want to set overall limits), but also for (b) the people on the front lines who are the ones actually making decisions, accepting risks, and taking actions to manage the risks. The guidance also has to explain the need to measure and report on whether the actions taken on the front lines aggregate to levels within organizational tolerances.”

Great idea, but for this one to work, you’d have to establish guidance around reward-taking, tolerance, etc., too.

5.) Norm wants “A change to the opinion provided by the external auditors, from one focusing on compliance with GAAP to one focusing on whether the financial reports filed with the regulators provide a true and fair view of the results of operations, the condition of the organization, and the outlook for the future.”

I’m going with “bad idea” on this one.  Accountants != entrepreneurs.  Despite all their longing for control, power, and self-importance.

6.)  For Norm, Regulators should receive ” An opinion by management on the effectiveness of the enterprise-wide risk management program. This could be based on the assessment of the internal audit function”

I’m confused, how is the internal audit function in any way at all related to the quality of decision making?  Assurance is *an* evidence, a confidence value for specific risk factors.  It seems that Norm is saying that assurance is *the* evidence in total.

Frankly, very few accountants have training or exposure to probability theory, decision theory, or complexity theory.  Until they *do*, my wish for 2012 is that CPAs  reserve judgement on people trying to use real methods to solve real problems.

7.) Norm wants:  A change in attitude of investor groups, focusing on longer-term value instead of short-term results.

AGREED and +1 to you Norm!

In 10.) a, Norm desires that “audit engagements should be prioritized based on the risk to the organization and the value provided by an internal audit project.”

ABSOLUTELY NOT.  Unless Audit engagements are to be prioritized by the faulty idea of “Inherent Risk”.

Example, as a risk manager – I may have relatively stable frequency and magnitude of operational losses.  They may fall into a “low” tolerance range established by an ERMC or something.  But even though I am doing a good job (or really lucky) I may really be concerned about the process enough to warrant a high frequency of audit.  There are just so many concerns about this sort of approach by an auditor (from a risk/actuary standpoint) that I can’t disagree more.

In point 11 Norm’s wish is “An improved understanding by the board and top management of the value of internal audit as a provider of assurance relative to governance, risk management”

Me too, but I don’t think Norm and I agree on that “value.”

Again, for a mature risk management group, the value of assurance is simply the establishment of confidence values for certain inputs.  And frankly, if the board and top management understood that, I’m not sure Norm would really want them too, because many times the assurance is really a reinforcement of confidence/certainty, and frankly is a job that can easily be done with a risk model that reduces SME bias.

Finally, Norm “would like to see the term “GRC” disappear”

AMEN.  To use the ISACA/Audit terminology, Compliance is just “a risk.”  To use risk terminology, Compliance is a factor that contributes to secondary or indirect losses.

So, I’m with you – I’d like to see GRC taken out behind the shed.  Where I differ is that’s not because it becomes coupled with risk management, but rather because for me compliance aligns better with the authoritarian world of audit rather than a discipline like risk whose goal is to reduce subjectivity, or a discipline like governance whose role is to optimize resource expenditures.

The One Where David Lacey's Article On Risk Makes Us All Stupider

In possibly the worst article on risk assessment I’ve seen in a while, David Lacey of Computerworld gives us the “Six Myth’s Of Risk Assessment.”  This article is so patently bad, so heinously wrong, that it stuck in my caw enough to write this blog post.  So let’s discuss why Mr. Lacey has no clue what he’s writing about, shall we?

First Mr. Lacey writes:

1. Risk assessment is objective and repeatable

It is neither. Assessments are made by human beings on incomplete information with varying degrees of knowledge, bias and opinion. Groupthink will distort attempts to even this out through group sessions. Attitudes, priorities and awareness of risks also change over time, as do the threats themselves. So be suspicious of any assessments that appear to be consistent, as this might mask a lack of effort, challenge or review.

Sounds reasonable, no?  Except it’s not alltogether true.  Yes, if you’re doing idiotic RCSA of Inherent – Control = Residual, it’s probably as such, but those assessments aren’t the totality of current state.

“Objective” is such a loaded word.  And if you use it with me, I’m going to wonder if you know what you’re talking about.  Objectivity / Subjectivity is a spectrum, not a binary, and so for him to say that risk assessment isn’t “objective” is an “of course!”  Just like there is no “secure” there is no “objective.”

But Lacey’s misunderstanding of the term aside, let’s address the real question: “Can we deal with the subjectivity in assessment?”  The answer is a resounding “yes” if your model formalizes the factors that create risk and logically represents how they combine to create something with units of measure.  And not only will the right model and methods handle the subjectivity to a degree that is acceptable, you can know that you’ve arrived at something usable when assessment results become “blindly repeatable.”  And yes, Virginia, there are risk analysis methods that create consistently repeatable results for information security.

2. Security controls should be determined by a risk assessment

Not quite. A consideration of risks helps, but all decisions should be based on the richest set of information available, not just on the output of a risk assessment, which is essentially a highly crude reduction of a complex situation to a handful of sentences and a few numbers plucked out of the air. Risk assessment is a decision support aid, not a decision making tool. It helps you to justify your recommendations.

So the key here is “richest set of information available” – if your risk analysis leaves out key or “rich” information, it’s pretty much crap.  Your model doesn’t fit, your hypothesis is false, start over.  If you think that this is a trivial matter for him to not understand, I’ll offer that this is kind of the foundation of modern science.  And mind you, this guy was supposedly a big deal with BS7799.  Really.

4. Risk assessment prevents you spending too much money on security

Not in practice. Aside from one or two areas in the military field where ridiculous amounts of money were spent on unnecessary high end solutions (and they always followed a risk assessment), I’ve never encountered an information system that had too much security. In fact the only area I’ve seen excessive spending on security is on the risk assessment itself. Good security professionals have a natural instinct on where to spend the money. Non-professionals lack the knowledge to conduct an effective risk assessment.

This “myth” basically made me physically ill.  This statement “I’ve never encountered an information system that had too much security” made me laugh so hard I keeled over and hurt my knee in the process by slamming it on the little wheel thing on my chair.

Obviously Mr. Lacey never worked for one of my previous employers that forced 7 or so (known) endpoint security applications on every Windows laptop.  Of course you can have too much !@#%ing security!  It happens all the !@#%ing time.  We overspend where frequency and impact ( <- hey, risk!) don’t justify the spend.  If I had a nickel for every time I saw this in practice, I’d be a 1%er.

But more to the point, this phrase (never too much security) makes several assumptions about security that are patently false.  But let me focus on this one:  This statement implies that threats are randomly motivated.  You see, if a threat has targeted motivation (like IP or $) then they don’t care about systems that offer no value in data or in privilege escalation.  Thus, you can spend too much on protecting assets that offer no or limited value to a threat agent.

5. Risk assessment encourages enterprises to implement security

No, it generally operates the other way around. Risk assessment means not having to do security. You just decide that the risk is low and acceptable. This enables organisations to ignore security risks and still pass a compliance audit. Smart companies (like investment banks) can exploit this phenomenon to operate outside prudent limits.

I honestly have no idea what he’s saying here.  Seriously, this makes no sense.  Let me explain.  Risk assessment outcomes are neutral states of knowledge.  They may feed a state of wisdom decision around budget, compliance, and acceptance (addressing or transferring, too) but this is a logically separate task.

If it’s a totally separate decision process to deal with the risk, and he cannot recognize this is a separate modeling construct, these statements should be highly alarming to the reader.  It screams “THIS MAN IS AUTHORIZED BY A MAJOR MEDIA OUTLET TO SPEAK AS AN SME ON RISK AND HE IS VERY, VERY CONFUSED!!!!”

Then there is that whole thing at the end where he calls companies that address this process illogically as “smart.”  Deviously clever, I’ll give you, but not smart.

6. We should aspire to build a “risk culture” across our enterprises

Whatever that means it sounds sinister to me. Any culture built on fear is an unhealthy one. Risks are part of the territory of everyday business. Managers should be encouraged to take risks within safe limits set by their management.

So by the time I got to this “myth” my mind was literally buzzing with anger.  But then Mr. Lacey tops us off with this beauty.  This statement is so contradictory to his past “myth” assertions, is so bizarrely out of line with his last statement in any sort of deductive sense, that one has to wonder if David Lacey isn’t actually an information security surrealist or post-modernist who rejects ration, logic, and formality outright for the sake of random, disconnected and downright silly approaches to risk and security management. Because that’s the only way this statement could possibly make sense.  And I’m not talking “pro” or “con” for risk culture here, I’m just talking about how his mind could possibly conceptually balance the concept that an “enterprise risk culture” sounds sinister vs. “Managers should be encouraged to take risks within safe limits set by their management” and even “I’ve never encountered an information system that had too much security.”

(Mind blown – throws up hands in the air, screams AAAAAAAAAAAAAAAAAHHhHHHHHHHHHHHHH at the top of his lungs and runs down the hall of work as if on fire)

See?  Surrealism is the only possible explanation.

Of course, if he was an information security surrealist, this might explain BS7799.