Time for an Award for Best Data?
Yesterday, DAn Kaminsky said “There should be a yearly award for Best Security Data, for the best collection and disbursement of hard data and cogent analysis in infosec.” I think it’s a fascinating idea, but think that a yearly award may be premature. However, what I think is sorta irrelevant, absent data. So I’m looking for data on the question, do we have enough good data to issue an award yearly?
Please nominate in the comments.
Also, please discuss what the criteria should be.
Tough call.
The best collection may be about data that are not themselves released, but are instead rolled up into aggregated or summarized data which are then most cogently analyzed.
I can see many of the “security reports” from WhiteHat, MSFT, Veracode and so on having precisely this characteristic.
Meanwhile, there are some collections of data that are released with comparatively little analysis, but are quite fine-grained. An example here would be DatalossDB (and perhaps OSVDB as well – I am not as familiar with it).
Maybe we need two award – best data source (which would include collection and disbursement) and most cogent analysis of data collected by the analyst.
In the analysis category, I’d nominate Googles “we don’t know what the pattern is but we can build a machine that learns it” approach to data, as exemplified by “Beyond Heuristics: Learning to Classify
Vulnerabilities and Predict Exploits”, http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en//pubs/archive/36738.pdf
There are so many pitfalls and potential fallacies when we approach security in what appears to us as a logical or formal way that the agnostic machine-learning approach may be the best that we know to date.
Bam!
eprint.iacr.org/2012/064.pdf