Security is about outcomes, not about process
Nearly a decade ago Bruce Schneier wrote “Security is a process, not a product.” His statement helped us advance as a profession, but with the benefit of hindsight, we can see he’s only half right. Security isn’t about technology.
Security is about outcomes, and our perceptions, beliefs and assurance about those outcomes.
Here’s a quick gut check: which would you rather tell the board or your customers? (1) “We had no security incidents last year, and aren’t sure why,” or (2) “Our customer database was pillaged 9 times, despite a cross-organizational investment in ISO 27001 which was aligned with our balanced scorecard and measured to be in the top quartile of all infosec programs?”
However people orient themselves around security, what they worry about is not “does the organization follow COBIT or ITIL?” but, “will they protect the information I’m giving them?”
Across the variety of orientations which exist within security, outcomes are what counts. Some examples:
- Compliance officers want to keep the CEO out of jail. All the process in the world is useful because when they’re not, they can talk about their plans for correcting that.
- Applied Researchers ask “did you pwn it?” They’re concerned with testing a hypothesis, which is “this system resists this type of attack”
- Law enforcement wants to catch the bad guy (or gal). Much of the friction between civil libertarians and law enforcement comes from a conflict about prioritization of goals.
We’ve focused on process because we have so little data on outcomes. People will talk about their training processes. But when you ask them, did that process work? no one wants to say.
For us to mature as a discipline and as part of the organizations we support, we must go from talking about what might happen to what does happen.
How about “We had no significant incidents, and are pretty sure why?”
Adam,
I’m onboard here but consider the bank that had no attempted robberies this year, or ever for that matter. All of the other banks have a safe, armed guard, silent alarms, dyepacks foe the stolen money, etc, and so do they.
If those countermeasures don’t get “used” in a year is it because they acted as a deterrent, or just no one wanted to rob that bank?
Maybe they even counted every tellers drawer at the end of each shift and the drawer always balanced.
They would measure things like:
– Number of drawers that balanced (not number of attempted thefts by tellers deterred)
– Number of times guards doing rounds were where they were supposed to be (not, how many burglars they stopped during their rounds)
Was all of that a waste of time? Process that didn’t result in a good outcome? Or, was it worth having done all of that? Was it worth it? All they are measuring is process….
Your “which would you rather tell the board or your customers?” question is a false comparison. In the first case (“We had no security incidents last year, and aren’t sure why”), the follow-up question would be “how do you know you haven’t had an incidents?” and the likely response would also be “we don’t know” or “nothing bad seemed to happen.” In any case, it’s rather unlikely that there’s any difference between either case, except that in the second you actually know what’s happening in your environment, where in the first you don’t have a clue and could be getting ravaged without knowing it.
You need to be very careful with this “managing to objectives” approach. It can quickly devolve into an “end justifies the means” discussion, and that could become quite counterproductive (do we really want corporations launching electronic attacks against foreign-based hackers?).
That being said, you’re absolutely right that objectives need to be established. “What’s our goal for security this week/month/year?” Those goals should directly correlate to business objectives, helping ensure the efficient and effective realization of those goals. In fact, James McGovern touched on this subject recently: http://duckdown.blogspot.com/2009/04/explaining-security-to-business-people.html
Good post.
I think you are confusing two problems:
1. People thinking security is a one-time implementation, and
2. The fact that we’re not good at success metrics
These are two different things, so “process” and “outcomes” are not on the same scale, with us needing to pick on or the other. They’re two different scales.
So, then is the question: What incentives do you use to attain the desired outcomes?
What’s your business model? If you haven’t grounded the security model against a set of threats to the business, then there is no way to measure the utility of outcomes. Problem is, it’s hard to learn the language of business and the language of security, and still find time for a life that’s not crime.
Security definitely isn’t just about outcomes; it is much more complex than that. Security is all about balancing enough effort (and cost), plus a reasonable margin for error, against the probability of a scenario occurring. The way that the effort is formalised and made repeatable is the process. And good process isn’t necessarily the same thing as an accreditation.
Sure, some people have no idea how or why they had no security incidents this year. Likewise, I’m sure that there are plenty of people that were pwned and still have no idea what happened (and more still that were pwned, but have yet to discover the happy event). The rest of us that do understand about the importance of good process, and how it strongly influences the possible outcomes, can get back to our coffee and finger-biscuits in the drawing room…
Martin…
Good way to focus the attention. It’s got me thinking that the bigger question is: If security is about outcomes, what do you audit?
Well, where do we go with that? Hmm..
For one, it severely devalues the “list of controls” audit standards like PCI – which I will lose no sleep over.
For two, it puts a more useful spin on “risk-based” controls like ISO 27001 – again, a good thing.
But, think about objective-based audits like SAS-70. Here you assert that you’ve protected a bunch of things and then assert that you have adequate controls to cover them. Then the auditors try to pick apart both the design (did you pick the right controls?) and the execution (are you really doing it). Sure, some bad auditors may fall back to a list of canned controls (COBIT), but not all auditors are stupid. Of course, this also puts the burden on people to actually read an audit report and look at the scope & objectives and then rate the overall value of the security program – a very good thing.
Ben, Daniel,
Thanks everyone for the great comments.. I’m still processing much of what’s been said.
Process and technology are also not on the same scale. Nevertheless, they’re both things you can invest in.
Martin,
I agree there’s complexity. My argument is that we invest in process to achieve outcomes, not to wallow in process.
More shortly. 🙂
You might also want to credit Shewart (1930s) and Deming (1950s) when discussing theories of security process versus outcomes. Or if you believe Wikipedia, Francis Bacon also gets a nod:
http://en.wikipedia.org/wiki/PDCA
Having thought about this some more, what activity of any substance _isn’t_ about outcomes? I understand what you’re saying of course, and I understand the importance of branding, but I wonder whether “security is about outcomes” is as vapid as “security is a process”.
Sorry, in a contrarian mood.
I’m not sure you aren’t making a false distinction. Aren’t outcomes just the measure of the process?
I would maintain that security is still a process (and certainly not a product), but that we as an industry need to put real development effort into appropriate measurement of those processes to ensure we can articulate the outputs/outcomes of the process.
Well-mannered place! Add to favorite
04O6F, I really like your site! j4qeu
04O6F, I really like your site! j4qeu
04O6F, I really like your site! j4qeu
04O6F, I really like your site! j4qeu
04O6F, I really like your site! j4qeu
04O6F, I really like your site! j4qeu