Doing Science With Near Misses

Last week at Art into Science, I presented “That was Close! Doing Science with Near Misses” (Slides as web page, or download the pptx.)

The core idea is that we should borrow from aviation to learn from near misses, and learn to protect ourselves and our systems better. The longer form is in the draft “That Was Close! Reward Reporting of Cybersecurity ‘Near Misses’Voluntary Reporting of Cybersecurity “Near Misses”

The talk was super-well received and I’m grateful to Sounil Yu and the participants in the philosphy track, who juggled so we could collaborate and brainstorm. If you’d like to help, by far the most helpful way would be to tell us about a near miss you’ve experienced using our form, and give us feedback on the form. Since Thursday, I’ve added a space for that feedback, and made a few other suggested adjustments which were easy to implement.

If you’ve had a chance to think about definitions for either near misses or accidents, I’d love to hear about those, in comments, in your blog (trackbacks should work), or whatever works for you. If you were at Art Into Science, there’s a #near-miss channel on the conference Slack, and I’ll be cleaning up the notes.

Image from the EHS Database, who have a set of near miss safety posters.

Boyd Video: Patterns of Conflict

John Boyd’s ideas have had a deep impact on the world. He created the concept of the OODA Loop, and talked about the importance of speed (“getting inside your opponent’s loop”) and orientation, and how we determine what’s important.

A lot of people who know about the work of John Boyd also know that he rarely took the time to write. His work was constantly evolving, and for many years, the work existed as scanned photocopies of acetate presentation slides.

In 2005, Robert Coram published a book (which I reviewed here and in that review, I said:

His writings are there to support a presentation; many of them don’t stand well on their own. Other writers present his ideas better than he did. But they don’t think with the intensity, creativity, or rigor that he brought to his work.

I wasn’t aware that there was video of him presenting, but Jasonmbro has uploaded approximately 5 hours of Boyd presenting his Patterns of Conflict briefing. The audio is not great, but it’s not unusable. There’s an easy to read version of that slide collection here. (Those slides are a little later than the video, and so may not line up perfectly.)

Seattle event: Ada's Books

Shostack threat modeling Adas

For Star Wars day, I’m happy to share this event poster for my talk at Ada’s Books in Seattle
Technical Presentation: Adam Shostack shares Threat Modeling Lessons with Star Wars.

This will be a less technical talk with plenty of discussion and interactivity, drawing on some of the content from “Security Lessons from Star Wars,” adapted for a more general audience.

Heaven Forbid the New York Times include Atheists

In “Is Your Religion Your Financial Destiny?,” the New York Times presents the following chart of income versus religion:
NYTimes Religion Income Graph

Note that it doesn’t include the non-religious, which one might think an interesting group as a control. Now, you might think that’s because the non-religious aren’t in the data set. But you’d be wrong. In the data set are atheists, agnostics and “nothing in particular.” That last includes 6.3% of the population as “secular unaffiliated” and another 5.8% as “religious unaffiliated.” Now, 6.3% is more than all non-Christian religions combined. Many of those non-Christian religions are shown in the graphic. Athiest, at 1.6%, is almost as large as Jewish, a major focus of the article, and 4 times larger than Hindus.

Now, you might also argue that athiests were left out because there were too few in the sample (as opposed to demographic data.) But there were 439 athiests, and 251 reform Jews.

Chris Wyspoal pointed out that atheists land after Hindus and Jews for 75k+ incomes.

All the news that’s fit to print, indeed.

How to Present

As I get ready to go to South Africa, I’m thinking a lot about presentations. I’ll be delivering a keynote and a technical/managerial talk at the ITWeb Security Summit. The keynote will be on ‘The Crisis in Information Security’ and the technical talk on Microsoft’s Security Development Lifecycle.

As I think about how to deliver each of these talks, I think about what people will want from each. From a keynote, there should be a broad perspective, aiming to influence the agenda and conversation for the day, the conference and beyond. For a technical talk, I’m starting from “why should we care” and sharing experiences in enough depth that the audience gets practical lessons they can apply to their own work.

Part of being a great presenter is watching others present, and seeing what works for them and what doesn’t. And part of it is watching yourself (painful as that is). Another part is listening to the masters. And in that vein, Garr Reynolds has a great post “Making presentations in the TED style:”

TED has earned a lot of attention over the years for many reasons, including the nature and quality of its short-form conference presentations. All presenters lucky enough to be asked to speak at TED are given 18-minute slots maximum (some are for even less time such as 3- and 6-minute slots). Some who present at TED are not used to speaking on a large stage, or are at least not used to speaking on their topic with strict time restraints. TED does not make a big deal publicly out of the TED Commandments, but many TED presenters have referenced the speaking guidelines in their talks and in their blogs over the years (e.g., Ben Saunders).

Ironically, he closes with:

Bill Gates vs. Bill Gates
Again, you do not have to use slides at TED (or TEDx, etc.), but if you do use slides, think of using them more in the style of Bill Gates the TEDster rather than Bill Gates the bullet point guy from the past. As Bill has shown, everyone can get better at presenting on stage.


I’ll be doing some of both. As both Reynolds and Bill understand, there are better and worse styles. Different styles work well for different people. There’s also a time and a place for each good style of presentation. Understanding yourself, your audience and goals are essential to doing any presentation well.

Of course, style only matters if you’re a professional entertainer, or have something interesting to say. I try hard to be in the latter category.

If you’re in Johannesburg, come see both talks. I’m looking forward to meeting new people, and would love to hear your feedback on either talk, either on the content or the style.

My Wolfram Alpha Demo

I got the opportunity a couple days ago to get a demo of Wolfram Alpha from Stephen Wolfram himself. It’s an impressive thing, and I can sympathize a bit with them on the overblown publicity. Wolfram said that they didn’t expect the press reaction, which I both empathize with and cast a raised eyebrow at.

There’s no difference, as you know, between an arbitrarily advanced technology and a rigged demo. And of course anyone whose spent a lot of time trying to create something grand is going to give you the good demo. It’s hard to know what the difference is between a rigged demo and a good one.

The major problem right now with Alpha is the overblown publicity. The last time I remember such gaga effusiveness it was over the Segway before we knew it was a scooter.

Alpha has had to suffer through not only its creator’s overblown assessments, but reviews from neophiles whose minds are so open that their occipital lobes face forward.

My short assessment is that it is the anti-Wikipedia and makes a huge splat on the fine line between clever and stupid, extending equally far in both directions. What they’ve done is create something very much like the computerized idiot savant. As much as that might sound like criticism, it isn’t. Alpha is very, very, very cool. Jaw-droppingly cool. And it is also incredibly cringe-worthily dumb. Let me give some examples.

Stephen gave us a lot of things that it can compute and the way it can infer answers. You can type “gdp france / germany” and it will give you plots of that. A query like “who was the president of brazil in 1930” will get you the right answer and a smear of the surrounding Presidents of Brazil as well.

It also has lovely deductions it makes. It geolocates your IP address and so if you ask it something involving “cups” it will infer from your location whether that should be American cups or English cups and give you a quick little link to change the preference on that. Very, very, clever.

It will also use your location to make other nice deductions. Stephen asked it a question about the population of Springfield, and since he is in Massachusetts, it inferred that Springfield, and there’s a little pop-up with a long list of other Springfields, as well. It’s very, very clever.

That list, however, got me the first glimpse of the stupid. I scanned the list of Springfields and realized something. Nowhere in that list appeared the Springfield of The Simpsons. Yeah, it’s fictional, and yeah that’s in many ways a relief, but dammit, it’s supposed to be a computational engine that can compute any fact that can be computed. While that Springfield is fictional, its population is a fact.

The group of us getting the demo got tired of Stephen’s enthusiastic typing in this query and that query. Many of them are very cool but boring. Comparing stock prices, market caps, changes in portfolio whatevers is something that a zillion financial web sites can do. We wanted more. We wanted our queries.

My query, which I didn’t ask because I thought it would be disruptive, is this: Which weighs more, a pound of gold or a pound of feathers? When I get to drive, that will be the first thing I ask.

The answer, in case you don’t know this famous question is a pound of feathers. Amusingly, Google gets it on the first link. Wolfram emphasizes that Alpha computes and is smart as opposed to Google just dumbly searching and collating.

I also didn’t really need to ask because one of the other people asked Alpha to plot swine flu in the the US, and it came up with — nil. It knows nothing about swine flu. Stephen helpfully suggested, “I can show you colon cancer instead” and did.

And there it is, the line between clever and stupid, and being on both sides of it. Alpha can’t tell you about swine flu because the data it works on is “curated,” meaning they have experts vet it. I approve. I’m a Wikipedia-sneerer, and I like an anti-mob system. However, having experts curate the data means that there’s nothing about the Springfield that pops to most people’s minds (because it’s pop culture) nor anything about swine flu. We asked Stephen about sources, and specifically about Wikipedia. He said that they use Wikipedia for some sorts of folk knowledge, like knowing that The Big Apple is a synonym for New York City but not for many things other than that.

Alpha is not a Google-killer. It is not ever going to compute anything that can be computed. It’s a humorless idiot savant that has an impressive database (presently some ten terabytes, according to the Wolfram folks), and its Mathematica-on-steroids engine gives a lot of wows.

On the other hand, as one of the people in my demo pointed out, there’s not anything beyond a spew of facts. Another of our queries was “17/hr” and Alpha told us what that is in terms of weekly, monthly, yearly salary. It did not tell us the sort of jobs that pay 17 per hour, which would be useful not only to people who need a job, but to socioeconomic researchers. It could tell us that, and very well might rather soon. But it doesn’t.

Alpha is an impressive tool that I can hardly wait to use (supposedly it goes on line perhaps this week). It’s something that will be a useful tool for many people and fills a much-needed niche. We need an anti-Wikipedia that has only curated facts. We need a computational engine that uses deductions and heuristics.

But we also need web resources that know about a fictional Springfield, and resources that can show you maps of the swine flu.

We also need tech reviewers who have critical faculties. Alpha is not a Google-killer. It’s also not likely as useful as Google. The gushing, open-brained reviews do us and Alpha a disservice by uncritically watching the rigged demo and refusing to ask about its limits. Alpha may straddle the line between clever and stupid, but the present reviewers all stand proudly on stupid.

All atwitter

In re-reading my blog post on twittering during a conference I realized it sounded a lot more negative than I’d meant it to.

I’d like to talk about why I see it as a tremendous positive, and will be doing it again.

First, it engages the audience. There’s a motive to pay close attention and share what you hear. They’re using their laptops for good, not evil.

Second, it multiplies the attention to the talk. The talk was standing room only, but the room held fewer than 100 people. The people who tweeted had 5,300 followers. Now, that’s total followers, not unique (does anyone have an easy way to calculate that?) It’s also unlikely that many of them were reading Twitter or read backscroll, but it seems like an ok guess to say that 200-500 people saw some mention of the talk on Twitter.

Third, it promotes the audience from passive to engaged (although that wasn’t a problem for my audience, I’ve seen it in other talks). They’re no longer just listeners, they’re interpreting, quoting, and generating additional content as we engaged around the ideas in the talk.

What chaotically emerged is larger than my talk. It’s a conversation.