Shostack + Friends Blog Archive


Data Not Assertions

There have already been a ton of posts out there about the Verizon DBIR Supplement that came out yesterday, so I’m not going to dive into the details, but I wanted to highlight this quick discussion from twitter yesterday that really sums of the value of the supplement and similar reports:

georgevhulme: I’m glad we have data to refute the “insiders conduct 80% of all attacks” mantra that has been repeated, ad nauseum for at least a decade

adamshostack: @alexhutton @georgevhulme yeah, but… Data, not assertions

This is so awesome, I can barely stand it. We’re actually starting to be able to make data based decisions as opposed to just asserting something is true because we believe it on faith or like the way it sounds.

“Data, not assertions” really sums up so much of what I was trying to get at in the the discussion on securosis last week about password changing time frames. Read the comments over there. It really shows how far we have yet to go.

12 comments on "Data Not Assertions"

  • Chris says:

    My gut tells me Adam is right.

  • rmogull says:

    I don’t know, Adam was kind of wrong the last time.

  • Russell says:

    I don’t know if the DBIR or supplement answers the question about prevalence of insider vs. outsider attackers.

    The Verizon data set seems to be skewed to companies who have experienced breaches of financial data or PII. I didn’t see much related to intellectual property theft. This skew may just be the nature of demand for forensic investigation and also the nature of Verizon’s service portfolio. It may be that many companies who experience IP theft never go through full forensic investigation, or they use a professional services firm other than Verizon.

    For a single data point at a large pharmaceutical company, see this post at Dark Reading:

    This single case doesn’t prove anything about relative frequency, either, but if you believe his statement about financial losses and frequency, it doesn’t take many incidents for insider breaches to add up to big $$:

    “I know of at least 15 other similar cases. The average monetary loss of the case I worked on was estimated at $350 million yearly”

    • Appendix A, lends more credence to what Verizon has been finding as well. It may not be perfect, but it’s a much larger sample set that maps nicely, so it gives us what seems like a pretty good idea.

      “I know of at least 15 other similar cases. The average monetary loss of the case I worked on was estimated at $350 million yearly”

      The thing about IP loses is that they are usually claimed to be very high (c.f. Kevin Mitnick) and yet strangely you don’t see those loses showing up very often on 10-Ks. Makes those lose numbers seem a little suspicious don’t you think?

  • Jon Robinson says:

    Every professional service firm should publish their data in a standard format and then we could get to the bottom of this.

  • @Jon Robinson

    It only we had a standard model for sharing this data….

  • Jon Robinson says:

    @david I’m hoping a standard model will develop organically since the people publishing their data will want to compare it efficiently, as noted in the supplement appendix (methodology section).

  • Dutch says:

    Publishing in a standard format as in the report or the data sets?

    Common Result Format (CRF) looks like someone started to tackle the problem, but it appears static since 2007.

    CRF :

  • Pete says:

    CSI report shows 15% vs. 14% insiders vs. outsiders, respectively. This, of course, begs the question – what about the other 71%?

  • adam says:


    Actually, it begs the question, why are you quoting the CSI report? 🙂

Comments are closed.