Counting In Background Checks
There’s some fascinating presentation of numbers in the BBC’s “Criminal records mix-up uncovered:”
Education Secretary Alan Johnson told the BBC only 0.03% of the nine million “disclosures” the agency makes had been wrong, so the issue had to be put “into context”.
He is so right! Let’s put those numbers in context, shall we?
The article says that 25,000 “unsuitable” people had not been hired, and 2,700 had incorrectly been labeled as criminals out of 9 million inquiries. So I think that the right context is that 2,700 of 27,700 “unsuitable” responses were “incorrect.” That’s a 9.7% error rate, not a .03% error rate. (We also have no numbers on how many real criminals, like Cleon Jones were incorrectly passed.)
I also don’t know if people are being notified of why their job application is rejected. It could be that more of those 25,000 “unsuitable” people are “suitable,” but they don’t know to appeal, or that the same bureaucracy that refuses to apologize for being “cautious” is also “cautiously” failing to correct real errors in their system.
Returning to the math, the same sort of math applies to things like the No-Fly list. A very low false positive rate snares lots of innocent people, while there are no real terrorists being caught. This is because there are an awful lot of innocent people, so flagging a very small fraction of them leads to lots of people flagged.
On the bright side, the British Government is at least measuring its failure rate.
Via Canadian Privacy Law blog, “Almost perfect accuracy still labels hundreds as criminals in the UK.”