Security, privacy, economics and unrelated topics since 2005
In “Conway’s Law: does your organization’s structure make software security even harder?,” Steve Lipner mixes history and wisdom:
As a result, the developers understood pretty quickly that product security was their job rather than ours. And instead of having twenty or thirty security engineers trying to “inspect (or test) security in” to the code, we had 30 or 40 thousand software engineers trying to create secure code. It made a big difference.
You know, I left, and now look what Word wants to suggest..
One of the recurring lessons from Petroski is how great engineers overcome not only the challenges of physical engineering: calculating loads, determining build orders, but they also overcome the real world challenges to their ideas, including financial and political ones. For example:
Many a wonderful concept, beautifully drawn by an inspired structural artist, has never risen off the paper because its cost could not be justified. Most of the great bridges of the nineteenth century, which served to define bridge building and other technological achievements for the twentieth century, were financed by private enterprise, often led by the expanding railroads. Engineers acting as entrepreneurs frequently put together the prospectuses, and in some cases almost single-handedly promoted their dreams to the realists. […] Debates over how to pay for them were common. (Engineers of Dreams: Great Bridge Builders and the Spanning of America, Henry Petroski)
Many security professionals have a hobby of griping that products get rushed to market, maybe to be secured later. We have learned to be more effective at building security in, and in doing so, reduce product costs and increase on-time delivery. But some products were built before we knew how to do that, and others are going to get built by companies who choose not to do that. And in that sense, Collin Greene’s retrospective, “Fixing Security Bugs” is very much worth your time. It’s a retrospective on the Vista security program from a pen-test perspective.
Hacking: Exciting.
Finding bugs: Exciting.
Fixing those bugs: Not exciting.
The thing is, the finish line for our job in security is getting bugs fixed¹, not just found and filed. Doing this effectively is not a technology problem. It is a communications, organizational² and psychology problem.
I joined Microsoft while the Vista pen test was finishing up, and so my perspective is complimentary. I’d like to add a few additional perspectives to his points.
First, he asks “is prioritization correct?” After Vista, the SDL team created security bug bars, and then later refined them to align with the MSRC update priorities. That alignment with the MSRC priorities was golden. It made it super-clear that if you didn’t fix this before ship, you were going to have to do an update later. As a security engineer, you need to align your prioritization to the all up delivery priorities. Having everything be “extremely critical,” “very critical,” or “moderately critical” means you don’t know what matters, and so nothing does.
Second, “why security matters” was still a fight to be fought in Vista. By Windows 7, security had completed its “move left.” The spec form contained sections for security and privacy. Threat model review was a gate for start of coding. These process changes happened while developers were “rebelling” against Vista’s “overweight” engineering process. They telegraphed that security mattered to management and executives. As a security engineer, you need to get management to spend time talking about how security is balanced with other priorities.
Third, he points out that escalating to a manager can feel bad, but he’s right: “Often the manager has the most context on priorities.” Management saying “get this fixed” is an expression of prioritization. If you’ve succeeded in your work on “why security matters,” then management will know that they need to reinforce that message. Bringing the issues to them, responsibly, helps them get their job done. If it feels bad to escalate, then it’s worth asking if you have full buy in on security.
Now, I’m talking about security as if it matters to management. More and more, that’s the case. Something in the news causes leadership to say “we have to do better,” and they believe that there are things that they can do. In part that belief is because very large companies have been talking about how to make it work. But when that belief isn’t there, it’s your job as an engineer to, as Petroski says, single-handedly promote your dreams to the realists. Again, Greene’s post is full of good ideas.
Lastly, not everything is a bug. I discussed vulnerabilities versus design recently in “Emergent Design Issues.”
(Photo: https://www.pexels.com/photo/black-and-brown-insect-37733/)
Yesterday, I got into a bit of a back and forth with Wendy Nather on threat modeling and the role of risk management, and I wanted to respond more fully.
So first, what was said:
(Wendy) As much as I love Elevation of Privilege, I don’t think any threat modeling is complete without considering probability too.
(me) Thanks! I’m not advocating against risk, but asking when. Do you evaluate bugs 2x? Once in threat model & once in bug triage?
(Wendy) Yes, because I see TM as being important in design, when the bugs haven’t been written in yet. 🙂
I think Wendy and I are in agreement that threat modeling should happen early, and that probability is important. My issue is that I think issues discovered by threat modeling are, in reality, dealt with by only a few of Gunnar’s top 5 influencers.
I think there are two good reasons to consider threat modeling as an activity that produces a bug list, rather than a prioritized list. First is that bugs are a great exit point for the activity, and second, bugs are going to get triaged again anyway.
First, bugs are a great end point. An important part of my perspective on threat modeling is that it works best when there’s a clear entry and exit point, that is, when developers know when the threat modeling activity is done. (Window Snyder, who knows a thing or two about threat modeling, raised this as the first thing that needed fixing when I took my job at Microsoft to improve threat modeling.) Developers are familiar with bugs. If you end a strange activity, such as threat modeling, with a familiar one, such as filing bugs, developers feel empowered to take a next step. They know what they need to do next.
And that’s my second point: developers and development organizations triage bugs. Any good development organization has a way to deal with bugs. The only two real outputs I’ve ever seen from threat modeling are bugs and threat model documents. I’ve seen bugs work far better than documents in almost every case.
So if you expect that bugs will work better then you’re left with the important question that Wendy is raising: when do you consider probability? That’s going to happen in bug triage anyway, so why bother including it in threat modeling? You might prune the list and avoid entering silly bugs. That’s a win. But if you capture your risk assessment process and expertise within threat modeling, then what happens in bug triage? Will the security expert be in the room? Do you have a process for comparing security priority to other priorities? (At Microsoft, we use security bug bars for this, and a sample is here.)
My concern, and the reason I got into a back and forth, is I suspect that putting risk assessment into threat modeling keeps organizations from ensuring that expertise is in bug triage, and that’s risky.
(As usual, these opinions are mine, and may differ from those of my employer.)
[Updated to correct editing issues.]
This week, the annual Symposium on Usable Privacy and Security (SOUPS) is being held on the Microsoft campus. I delivered a keynote, entitled “Engineers Are People Too:”
In “Engineers Are People, Too” Adam Shostack will address an often invisible link in the chain between research on usable security and privacy and delivering that usability: the engineer. All too often, engineers are assumed to have infinite time and skills for usability testing and iteration. They have time to read papers, adapt research ideas to the specifics of their product, and still ship cool new features. This talk will bring together lessons from enabling Microsoft’s thousands of engineers to threat modeling effectively, share some new approaches to engineering security usability, and propose new directions for research.
A fair number of people have asked for the slides, and they’re here: Engineers Are People Too.
In my work blog: “Announcing Elevation of Privilege: The Threat Modeling Game.”
After RSA, I’ll have more to say about how it came about, how it helps you and how it helps more chaos emerge. But if you’re here, you should come get a deck at the Microsoft booth (1500 row).
In “U-Prove Minimal Disclosure availability,” Kim Cameron says:
This blog is about technology issues, problems, plans for the future, speculative possibilities, long term ideas – all things that should make any self-respecting product marketer with concrete goals and metrics run for the hills! But today, just for once, I’m going to pick up an actual Microsoft press release and lay it on you. The reason? Microsoft has just done something very special, and the fact that the announcement was a key part of the RSA Conference Keynote is itself important.
Further, Charney explained that identity solutions that provide more secure and private access to both on-site and cloud applications are key to enabling a safer, more trusted enterprise and Internet. As part of that effort, Microsoft today released a community technology preview of the U-Prove technology, which enables online providers to better protect privacy and enhance security through the minimal disclosure of information in online transactions. To encourage broad community evaluation and input, Microsoft announced it is providing core portions of the U-Prove intellectual property under the Open Specification Promise, as well as releasing open source software development kits in C# and Java editions. Charney encouraged the industry, developers and IT professionals to develop identity solutions that help protect individual privacy.
Kim then goes on to analyze the announcement, which is a heck of an important one.
Disclaimer: I work for Microsoft, and am friends with many of the people involved. I still think this is tremendously important.
At Microsoft, there’s a very long history of ‘eating your own dogfood’ or using the latest and greatest daily builds. Although today, people seem to use the term “self-host,” which seems evidence that they don’t do either.
Eating your own dogfood gives you a decent idea of when it starts to taste ok, which is to say, ready for customers to see in some preview form.
Apropos of which, there’s a really interesting post at the Inkling blog, “Pay for your own dog food:”
Using your own product comes with a ton of benefits, because you become your own customer. The quality of your product likely increases because you can’t ignore it’s problems. They aren’t just your customers problems. They are your problems.
…
We’ve gotten in the habit of actually taking out our own credit card and using it on our own account sign up page. Yes, it’s a bit silly when the credit card processing takes some money off the top. But it makes the feeling very real that you are paying for this, and now it’s an expense just like it’s going to be an expense for your clients.
Three years and three days ago I announced that “I’m Joining Microsoft.” While I was interviewing, my final interviewer asked me “how long do you plan to stay?” I told him that I’d make a three year commitment, but I really didn’t know. We both knew that a lot of senior industry people have trouble finding a way to be effective in Microsoft’s culture.
So I wanted to pipe up and say I’m having a heck of a lot of fun, and have found places and ways to be effective. I’m getting to develop and share things like our SDL Threat Modeling Tool, and I get to be very transparent about the drivers and decisions that shape it. I’ve got some even cooler stuff in the pipeline, which I’m hoping will be public in the next year or so. My management (which has shifted a little) is supportive of me having two external blogs.
It’s been a heck of a ride so far. Dennis Fisher asked a great question to close this Hearsay Podcast, which is what surprised me the most? I was a little surprised by the question, but I’m going to stand by my answer, which is the intensity and openness of internal debate, and how it helps shape the perception that we’re all reading from the same script. It’s because we’ve seen the debate play out, with really well-informed participants, and remember which points were effective.
I can’t wait to see what happens in the next three years.
While I was running around between the Berkeley Data Breaches conference and SOURCE Boston, Gary McGraw and Brian Chess were releasing the Building Security In Maturity Model.
Lots has been said, so I’d just like to quote one little bit:
One could build a maturity model for software security theoretically (by pondering what organizations should do) or one could build a maturity model by understanding what a set of distinct organizations have already done successfully. The latter approach is both scientific and grounded in the real world, and is the one we followed.
It’s long, but an easy and worthwhile read if you’re thinking of putting together or improving your software security practice.
Incidentally, my boss also commented on our work blog “Building Security In Maturity Model on the SDL Blog.”
