Are Security "Best Practices" Unethical?
Anton Chuvakin’s been going old school. Raising the specter of “risk-less” security via best practices and haunting me like the ghost of blog posts past. Now my position around best practices in the past has been that they are, to use Jack Jones’ phrase, Infosec “shamansim”. We do these things because our forefathers do them, because the tribe up the river does them, and/or because if we don’t the thunder gods (hackers) will get us, not because we have any formal evidence that they are “best”. Now that said, these suggested practices I keep seeing are not completely uninformative, after all there’s a reason people suggest the things they suggest. But being Friday and all, maybe we could talk a bit about why we don’t really have best practices, why Anton’s concept of risk-less security is a fallacy, and what we call best practices? Yeah, I think we could even argue that they’re unethical.
DO WE EVEN HAVE “BEST” PRACTICES?
First, why don’t we head over to that all-authoritative source, Wikipedia, for a definition of Best Practice.
A Best practice is a technique, method, process, activity, incentive or reward that is believed to be more effective at delivering a particular outcome than any other technique, method, process, etc. The idea is that with proper processes, checks, and testing, a desired outcome can be delivered with fewer problems and unforeseen complications. Best practices can also be defined as the most efficient (least amount of effort) and effective (best results) way of accomplishing a task, based on repeatable procedures that have proven themselves over time for large numbers of people.
Despite the need to improve on processes as times change and things evolve, best-practice is considered by some as a business buzzword used to describe the process of developing and following a standard way of doing things that multiple organizations can use for management, policy, and especially software systems.
As the term has become more popular, some organizations have begun using the term “best practices” to refer to what are in fact merely ‘rules’, causing a linguistic drift in which a new term such as “good ideas” is needed to refer to what would previously have been called “best practices.”
Now whether or not you agree that Wikipedia is useful, I think that passage serves our discussion today. I have long said that if we were to use the word “best”, that would require us to know “not best”. So considering:
Best practices can also be defined as the most efficient (least amount of effort) and effective (best results) way of accomplishing a task, based on repeatable procedures that have proven themselves over time for large numbers of people.
Maybe you could tell me about “efficient” and “effectiveness” – and how you would compare a “best practice” to an “almost best practice” without hand waving and resorting to a “just because I said so” argument. I don’t think you could, our (the infosec industry) measurement capabilities aren’t there yet, and so though while we have various reports and studies that suggest certain things are more effective than others, and we’re making great progress, Exhibit A today is:
Exhibit A – We don’t really have “best” practices – yet.
I could see saying “recommended” practices, “suggested” practices, or even based on some studies out there “essential” practices, sure. But we don’t have the sophistication of “best/not best” and I would argue that we, as an industry, are guilty of
using the term “best practices” to refer to what are in fact merely ‘rules’, causing a linguistic drift in which a new term such as “good ideas” is needed to refer to what would previously have been called “best practices.”
CAN YOU HAVE RISK-LESS SECURITY?
I’m not sure of what desire is driving Anton towards risklessness, but it’s folly. Here’s why: as soon as you consider a control, you’re considering risk. Impact comes easily and instinctively, likelihood in a similarly quick and instantaneous fashion. Otherwise, to use Peter Tippett’s examples, we’d all have titanium seatbelts and falling asteroid protectors on our cars. Now the hitch is, we (human brains) are notoriously bad at likelihood without the methods of models and math. But my point still stands – when we think security (and especially we, the people paid to think security) consider an asset we want to protect, we’re already doing a risk assessment, albeit in manner that just isn’t as structured as one that might use a formal model.
Exhibit B – So really, we’re *all* doing risk, and when we have these discussions of “use risk or don’t use risk” – all we’re really arguing about is how much cognitive bias we’re allowing into our assessment.
If you want to be on the side of the debate that suggests that lots of cognitive bias is a good thing, so be it. There’s no rational way I can argue with you – have a nice day, thanks and come again. But if the role and purpose of risk modeling is to remove bias and provide a state of knowledge – then I have to ask you,
IS IT ETHICAL TO USE A RISK-LESS APPROACH?
Strong charge, I know. And certainly an Incomplete thought that needs more discussion, but bear with me while I reason this out for us. Take this quote from Donn Parker against a risk-based approach:
risk reduction justification makes it too easy to accept security vulnerabilities in exchange for other benefits.”
The problem with this statement is that the economic decision of butter vs. guns *is not ours to make*. In all the CISSP study guides sitting on my library shelves, the premise is offered that the asset does not belong to us, security, but rather to the data owner. So with all due respect, that’s not your call to make! Not mine, not Donn’s, not yours. It is the responsibility of the business owner. Our responsibility is to do the best job we can informing them of the (wait for it) consequences of accepting (or not-accepting) a security vulnerability.
So if our job is to inform, not decide – and if risk analysis is the best way we have of reducing cognitive bias – then is resorting to a “do it because I say so” “best” practice approach ethical? Because what you’re essentially doing is purposefully withholding information from the data owner in order to force them to accept your risk tolerance. A “best” practice approach, without reference-able data or analysis to back it up is not just an act of hubris, it is deceit.
Exhibit C – If our job is to help the data owner make informed decisions, calling what we do in our industry a “best” practice is not entirely truthful, and not using a risk model that removes bias and expresses uncertainty is not doing the best job you can.
CAVEATS GALORE
1.) There’s nothing wrong with real best practices. In a sense, that’s what the New School is all about, inspiring us to arrive at things like (reasonably) measurable efficiency and effectiveness. After all, a best practice could be said to be the current standing, well-tested hypothesis or theory.
2.) There are plenty of times when we don’t have to go and do a formal risk assessment. My goal here is not “spreadsheets for the masses”, it’s rational expressions of risk management and helping people make the best decision they can with the information available at the time of the decision.
3.) Admittedly, 95% of the “likelihood” determinations out there in our industry, in the words of my 11 year old, make me want to throw a sidewalk pizza. And I could see a response post where you might claim that bad likelihood models are just as unethical, and that there’s so much uncertainty in our estimates, we are many times doing a disservice. Again, I’m not saying we’re in a perfect world yet, I’m suggesting that a good risk model removes bias and expresses uncertainty.
Similarly, the enemy here is not subjectivity – everything we do is subjective and/or relative. The enemy here is not removing all the subjectivity you can and not expressing uncertainty in the results. Our industry’s “best practices” don’t do that yet. Let me encourage you to express the limitations in everything – it is the pathway of critical thinking we must travel.
4.) I’m not accusing anyone, nor do I think anyone is really being purposefully unethical or deceitful. I can say that Anton and Donn are really good, smart, and upstanding people in as much as I’ve had the pleasure of knowing them. I’m just trying to caution us all about the consequences of what we do and how we approach our profession.
I’m fond of telling my clients to, “Forget about best practices. Let’s use good practices. Once we’ve been doing good practices for a while, we may or may not find that they were the best ones. But about one thing we can be fairly certain. Many (not all) of your current practices are bad ones.”
“Best Practices” feels a lot like “World Class”. Most folks want and need “good enough”. I wonder if we push for best practices and world class because doing so usually sets up unachievable goals. Best Practices generate plenty of consulting business because you almost never get there. Good enough practices are easier for an organization but make for fewer consulting hours.
Amen, Alex! One example of “best practice” is password policy, e.g.: http://www.schneier.com/blog/archives/2009/08/password_advice.html
But such password policies have usability problems:
“Security companies and IT people constantly tells us that we should use complex and difficult passwords. This is bad advice, because you can actually make usable, easy to remember and highly secure passwords. In fact, usable passwords are often far better than complex ones. ” http://www.baekdal.com/articles/Usability/password-security-usability/
But has anyone tested alternative password policies in realistic settings (i.e. real users or test users, real systems, real workloads) and evaluated them by measuring security outcomes (i.e. breaches, incidents, near-misses)? If so, then you deserve to call your preferred policy a “best practice”. Otherwise, it’s just folklore. Might be useful, might not.
We could probably assemble good evidence to support a list of “worst practices” (e.g. leaving web servers in default configuration, never patching critical software) from forensic analysis of breaches. But just because something is better than “worst practice” doesn’t make it a “best practice”.
When speaking with clients or upper management, I generally refer to “standard practices”. The fact that ‘everyone else does it’ just makes it a de facto standard, not necessarily optimal. From a statistical perspective, unless everyone does the same thing, I don’t think we can all do the “best”.
Hi, Alex –
I couldn’t agree more – this issue is also a pet peeve of mine, although I am learning to keep my mouth shut more than I used to.
Best implies a comparison of at least 3 alternatives, right? good, better, best?
Absent data, it’s pretty hard to make such a comparison in any kind of meaningful way.
Standard practice is problematic for me, maybe because in medicine – part of my background – a standard of care implies a significant legal obligation.
Also I don’t see the relevance of saying that something would be standard just because everyone did it. Millions of lemmings jump off cliffs – I guess that by the time they figure out what they have done, terminology is almost irrelevant 🙂
I don’t have a problem with good practice. Some people do have long and deep experience, and it’s important to respect that, even if it’s “anecdotal” evidence.
Patrick
Dallas
Patrick,
I totally agree. Best Practices are like checklists – they’re fine with significant experience/wisdom. Pre-flight checklists work because we know what makes an airplane fall out of the sky. We don’t really know what keeps a network “in the air”.
A “best practice” is naught but a management tool.
Pick carefully among those presented to you by readers of airline magazines, choosing those that make sense for achieving the one metric that matters: does sufficient revenue (or other measure of organizational success) continue?
Then manage users and management with the “best practice” you and they have selected to achieve that goal.
Also, don’t hesitate to feed externally imposed “best practices” through a challenge/BS-filter. The whole password complexity thing is a case in point. Passwords are not compromised in the majority here by being non-complex. They’re mostly compromised by being phished or by being logged by trojans/malware. Our “best practice” for passwords thus have complexity and change requirements tailored to those threats, not to what too many others are still doing.
(I’m going to stay out of the whole “security metrics” thing beyond noting that it’s a wonderful goal for those with far more resources to do science on their users than most of us can afford. 🙂
Finally, your assertion that the economic decision of butter vs. guns is not our call to make is incorrect (I guess I disagree with more of CISSP than I knew). We do make that call; it’s what our management pays us for. Even better, their charge to us is to find ways to have both together at reduced expense. That’s a competitive advantage: employees are more productive because they both find it easier to get their work done, and we’ve reduced the hit from compromises and cleanup.
Hi Richard!
RE: Guns vs. Butter – I suppose it depends at what level you’re talking about.
For “cost of downtime vs. vulnerability” or “$5 million for security or $5 million for marketing” sort of decisions, I’d argue those are business decisions that security shouldn’t be making.
If it is “this is my $15 million security budget to figure out how much to P/D/R – then many times, yes, that is a CISO’s decision – but again I would qualify that by saying that the CISO’s job is to spend that money within the *context* of the risk tolerance of the business owners.
Exactly; I’ve always appreciated that the Information Security Forum (ISF) publishes their Standard of GOOD Practice — not BEST. Security is a business decision.
Alex, did you just kick me in the balls 🙂
In any case, the response is coming!