DHS to Survey Cybercrime
In what they hope will become the premier measure of national
cybercrime statistics, officials at the Homeland Security and Justice
departments plan to survey 36,000 businesses this spring to examine
the type and frequency of computer security incidents.
This is a really exciting development. DHS seems to be taking a good approach, and in a preliminary survey, got over a 40% response rate. Read the Federal Computer Week article. Back in October, I wrote:
There’s a third role, which I think that the government might be able to play well, and that is helping us collect information. Today, being broken into is seen as an embarrassing failure. In many ways, it is. But, given the number of cases the cops are dealing with, its also very common. Much more common than you’d think. The federal government, either the FBI or DHS ought to be collecting crime statistics.
This has the potential to be very useful.
I would be very interesting in hearing how they drew their sample, as well as whether they have outsourced development of the survey instrument itself, and the actual administration of the survey. If all they are doing is mailing out an envelope and hoping it comes back, their response rate is going to suffer (40% is not good at all — they should be able to get nearly double that). From the rather light FCW article, they certainly seem to understand these issues, but as in so many other things, the devil is in the details.
I see what you’re saying, but I think 40% is much better than the CSI/FBI survey gets. I downloaded and skimmed the latest, from http://www.gocsi.com/forms/fbi/csi_fbi_survey.jhtml and can’t find what their response rate was. On at least some questions, their rate of response by people otherwise willing to answer the survey was under 60%.
I suppose whether a given response rate is “good enough” depends on what you are trying to measure, and whether you have any reason to think that there’s non-response bias involved. However, when you’re talking 60% non-response, it would seem to be a foregone conclusion.
Anyway, w.r.t. CSI, they could have 100% completion and the stats would be meaningless, since their sample is (IIRC) drawn entirely from security professional organization members, and even within that class, the respondents are self-selected.
In any case, this should not be a difficult nut to crack, in principle. The US government conducts surveys of businesses all the time, and is capable of obtaining quality samples and high response rates in which academics justly have confidence. I don’t know what the DHS budget for data collection is, but if I did, and had some idea of where the sample was coming from, I’d know how much to be excited about this thing.
There is definitely more of a hunger for real data “out there” then there used to be, and some players with $$ are getting into the act. Whether they can displace the self-serving pseudo-science remains, I suppose, to be seen.