RiskIT – Does ISACA Suffer From Dunning-Kruger?
Just to pile on a bit….
You ever hear someone say something, and all of the sudden you realize that you’ve been trying to say exactly that, in exactly that manner, but hadn’t been so succinct or elegant at it? That someone much smarter than you had already thought about the subject a whole lot and there’s actually formal studies and definitions and so forth, and you feel dumb because there’s no way you could have actually googled for the subject in that way, but there it is on wikipedia in black and white? Happens to me all the time.
Was reading the following from the NYT this morning on the Dunning-Kruger effect, and had a little bit of synchronicity when I realized that my entire problem with certifying people about risk and controls has to do with exactly this subject. My issue with ISACA and CRISC is that because I know that there’s so much that I don’t know, and indeed – that we, the infosec industry does not know, and so in my mind if we wanted to rationally, ethically “certify” someone about risk and controls as a domain expert, about the only thing we can do is to test that they are aware of their (and our) limitations. To do otherwise seems rather irrational to me.
TAKE ALEX’S “APPARENTLY OK” RISK PROFESSIONAL EXAM:
A dozen years or so ago, I was PM for a firewall product that had to go through certification. The certification involved several different tests, but at the time the key to certification was simply that your firewall would not pass packets when set in default-deny state. And it cost a lot to certify. This upset me to no end, mainly because I’d have rather spent the $50k certification cost on building a new GUI.
Knowing my frustration, one of the engineering team printed out Marcus Ranum’s “Apparently OK” firewall certification and taped to one of our boxes and congratulated me on getting certification.
With that spirit in mind, and with all apologies to Marcus, let me present Alex Hutton’s Apparently OK Risk Professional Certification Exam! Because frankly, the problem isn’t with “using risk management” – no I’m still a very big proponent of that. The problem is that risk analysis is steeped in critical thinking, and not identifying uncertainty is, well, less than professional in my opinion.
THE ALEX HUTTON “APPARENTLY OK” RISK PROFESSIONAL EXAM
Dear Prospective Risk Professional,
Congratulations on deciding to enter the exciting field of Information Risk Management! Your journey will be confusing, frustrating, and if you get off on performing sisyphean tasks, rewarding.
To achieve a state of “APPARENTLY OK” we ask that you take the following exam. The exam is one question, and you have three (3) minutes to answer it.
For all the risk assessment methodologies inventoried by ENISA (http://rm-inv.enisa.europa.eu/rm_ra_methods.html) and for RiskIT, please tell us the how the assessment methodology is fundamentally incapable of delivering the results claimed.
BONUS: Doing so in a total of two sentences.
There are multiple right answers.