Anton Chuvakin’s been going old school. Raising the specter of “risk-less” security via best practices and haunting me like the ghost of blog posts past. Now my position around best practices in the past has been that they are, to use Jack Jones’ phrase, Infosec “shamansim”. We do these things because our forefathers do them, because the tribe up the river does them, and/or because if we don’t the thunder gods (hackers) will get us, not because we have any formal evidence that they are “best”. Now that said, these suggested practices I keep seeing are not completely uninformative, after all there’s a reason people suggest the things they suggest. But being Friday and all, maybe we could talk a bit about why we don’t really have best practices, why Anton’s concept of risk-less security is a fallacy, and what we call best practices? Yeah, I think we could even argue that they’re unethical.
DO WE EVEN HAVE “BEST” PRACTICES?
First, why don’t we head over to that all-authoritative source, Wikipedia, for a definition of Best Practice.
A Best practice is a technique, method, process, activity, incentive or reward that is believed to be more effective at delivering a particular outcome than any other technique, method, process, etc. The idea is that with proper processes, checks, and testing, a desired outcome can be delivered with fewer problems and unforeseen complications. Best practices can also be defined as the most efficient (least amount of effort) and effective (best results) way of accomplishing a task, based on repeatable procedures that have proven themselves over time for large numbers of people.
Despite the need to improve on processes as times change and things evolve, best-practice is considered by some as a business buzzword used to describe the process of developing and following a standard way of doing things that multiple organizations can use for management, policy, and especially software systems.
As the term has become more popular, some organizations have begun using the term “best practices” to refer to what are in fact merely ‘rules’, causing a linguistic drift in which a new term such as “good ideas” is needed to refer to what would previously have been called “best practices.”
Now whether or not you agree that Wikipedia is useful, I think that passage serves our discussion today. I have long said that if we were to use the word “best”, that would require us to know “not best”. So considering:
Best practices can also be defined as the most efficient (least amount of effort) and effective (best results) way of accomplishing a task, based on repeatable procedures that have proven themselves over time for large numbers of people.
Maybe you could tell me about “efficient” and “effectiveness” – and how you would compare a “best practice” to an “almost best practice” without hand waving and resorting to a “just because I said so” argument. I don’t think you could, our (the infosec industry) measurement capabilities aren’t there yet, and so though while we have various reports and studies that suggest certain things are more effective than others, and we’re making great progress, Exhibit A today is:
Exhibit A – We don’t really have “best” practices – yet.
I could see saying “recommended” practices, “suggested” practices, or even based on some studies out there “essential” practices, sure. But we don’t have the sophistication of “best/not best” and I would argue that we, as an industry, are guilty of
using the term “best practices” to refer to what are in fact merely ‘rules’, causing a linguistic drift in which a new term such as “good ideas” is needed to refer to what would previously have been called “best practices.”
CAN YOU HAVE RISK-LESS SECURITY?
I’m not sure of what desire is driving Anton towards risklessness, but it’s folly. Here’s why: as soon as you consider a control, you’re considering risk. Impact comes easily and instinctively, likelihood in a similarly quick and instantaneous fashion. Otherwise, to use Peter Tippett’s examples, we’d all have titanium seatbelts and falling asteroid protectors on our cars. Now the hitch is, we (human brains) are notoriously bad at likelihood without the methods of models and math. But my point still stands – when we think security (and especially we, the people paid to think security) consider an asset we want to protect, we’re already doing a risk assessment, albeit in manner that just isn’t as structured as one that might use a formal model.
Exhibit B – So really, we’re *all* doing risk, and when we have these discussions of “use risk or don’t use risk” – all we’re really arguing about is how much cognitive bias we’re allowing into our assessment.
If you want to be on the side of the debate that suggests that lots of cognitive bias is a good thing, so be it. There’s no rational way I can argue with you – have a nice day, thanks and come again. But if the role and purpose of risk modeling is to remove bias and provide a state of knowledge – then I have to ask you,
IS IT ETHICAL TO USE A RISK-LESS APPROACH?
Strong charge, I know. And certainly an Incomplete thought that needs more discussion, but bear with me while I reason this out for us. Take this quote from Donn Parker against a risk-based approach:
risk reduction justification makes it too easy to accept security vulnerabilities in exchange for other benefits.”
The problem with this statement is that the economic decision of butter vs. guns *is not ours to make*. In all the CISSP study guides sitting on my library shelves, the premise is offered that the asset does not belong to us, security, but rather to the data owner. So with all due respect, that’s not your call to make! Not mine, not Donn’s, not yours. It is the responsibility of the business owner. Our responsibility is to do the best job we can informing them of the (wait for it) consequences of accepting (or not-accepting) a security vulnerability.
So if our job is to inform, not decide – and if risk analysis is the best way we have of reducing cognitive bias – then is resorting to a “do it because I say so” “best” practice approach ethical? Because what you’re essentially doing is purposefully withholding information from the data owner in order to force them to accept your risk tolerance. A “best” practice approach, without reference-able data or analysis to back it up is not just an act of hubris, it is deceit.
Exhibit C – If our job is to help the data owner make informed decisions, calling what we do in our industry a “best” practice is not entirely truthful, and not using a risk model that removes bias and expresses uncertainty is not doing the best job you can.
1.) There’s nothing wrong with real best practices. In a sense, that’s what the New School is all about, inspiring us to arrive at things like (reasonably) measurable efficiency and effectiveness. After all, a best practice could be said to be the current standing, well-tested hypothesis or theory.
2.) There are plenty of times when we don’t have to go and do a formal risk assessment. My goal here is not “spreadsheets for the masses”, it’s rational expressions of risk management and helping people make the best decision they can with the information available at the time of the decision.
3.) Admittedly, 95% of the “likelihood” determinations out there in our industry, in the words of my 11 year old, make me want to throw a sidewalk pizza. And I could see a response post where you might claim that bad likelihood models are just as unethical, and that there’s so much uncertainty in our estimates, we are many times doing a disservice. Again, I’m not saying we’re in a perfect world yet, I’m suggesting that a good risk model removes bias and expresses uncertainty.
Similarly, the enemy here is not subjectivity – everything we do is subjective and/or relative. The enemy here is not removing all the subjectivity you can and not expressing uncertainty in the results. Our industry’s “best practices” don’t do that yet. Let me encourage you to express the limitations in everything – it is the pathway of critical thinking we must travel.
4.) I’m not accusing anyone, nor do I think anyone is really being purposefully unethical or deceitful. I can say that Anton and Donn are really good, smart, and upstanding people in as much as I’ve had the pleasure of knowing them. I’m just trying to caution us all about the consequences of what we do and how we approach our profession.