Evaluating Security

The study, published in the January issue of the journal Emerging Infectious Diseases, concluded that the estimated $7.55 million spent on [SARS] screening at several Canadian airports failed to detect one case of the disease.

“Sometimes what seems like a reasonable thing to do doesn’t turn out that way,” the report’s lead author, Dr. Ronald St. John, told the Canadian Press.

Reports the CBC. (Kudos to the CDC for making its articles available as free web pages.) And wouldn’t it be nice if all security measures got this sort of treatment?

370,000 Absconders

Buried in this story about tracking illegal immigrants is the interesting item that as of early 2003, of 6,000 Muslims who absconded within the US after being told to leave the country, only 38 percent had been found. That left over 3,500 still at large. How many have been caught since then? Where are the others? What are they doing? From the Washington Post, “Tracking Down Immigrant Fugitives,” with thanks to Jeffrey Imm:

Writes JihadWatch. Tracking down and deporting these 3,500 people should be a priority. (I think it should be a higher priority than, say, searching everyone who buys a last minute ticket.) Why can’t the squad catch these folks? I’d like to pull some other quotes out of the article:

The reasons for the difficulty were clear as Smith’s team glided through the slumbering neighborhoods of Hyattsville. The officers had three targets in addition to Kabert. They had tried to pinpoint the immigrants by scouring real estate and other records. But the absconders left few paper trails.

The search for the absconders wasn’t supposed to be this difficult. When the program was announced in December 2001, officials said they would put the absconders’ names into FBI’s National Crime Information Center database. That would allow local and state police to identify whether people they stopped for routine infractions were on the list.

It was Etienne Kabert. He meekly led the officers to his apartment, where he had ID cards with various names and birth dates, as well as a French passport he had acquired years ago.

A reliance on databases over undercover or investigative work seems to be a hallmark of modern American thinking. But the reality is anyone can get a variety of ID cards; the immigration and drinking laws make it impossible to stamp out fraudulent ID issuance. (As I’ve discussed at length in talks like “Identity and Economics: Terrorism and Privacy” (Also available in Powerpoint or PDF). What good is a massive database of offenders when nearly everyone is a criminal, and any serious criminal can easily get new ID? If we start doing that, we’ll have the highest prison population in the world.

Ratty Signals

So, we have a security signal that’s available, but not used. Why might that be? Is the market in-efficient, or are there real limitations that I missed?

There are a few things that jump to mind:

  1. Size of code issues. More code will produce a longer report. Rats produces a line count, but doesn’t issue numbers in terms of say “8 issues per KLOC.”
  2. The severity of issues raised. How do you compare the low, medium and high severity issues? RATS doesn’t help with this.
  3. Ian Grigg mentioned a real instance of the perverse incentive to make changes to shut up compiler warnings.

So it seems that the market is reasonably efficient, and that RATS would make a poor signal, on difficulty of evaluating it.