Threat Modeling with Questionnaires

This post comes from a conversation I had on Linkedin with Clint Gibler. He wrote:

One challenge I’ve heard from a number of companies is that, with say 3-5 AppSec engineers supporting 500 – 1000 devs, you can’t TM every story, or even every epic. So what do you focus on?

The high risk / most critical things. But what are those? It’s not always easy to have visibility or even awareness of everything being built in fast moving, complex, large environments.

One method discussed in a few talks (and I reference in mine) is having devs fill out self-service security questionnaires that provide some detail about the purpose of the service / new feature, what sensitive data it touches, potentially dangerous functionality it might have, etc. so that security engineers can do a meaningful TM of those high risk apps.

As always, I’m picking this up to learn, not to criticize Clint for what he wrote. It is worthwhile to start by illuminating Clint’s mental model of technical threat modeling work, where he implicitly characterize threat modeling as a heavyweight consulting engagement by appsec engineers. In that model, there are probably deliverables like a DFD and a list of threats. And while that’s one fine model, there are other models, such as every engineer threat models, and the approach is lightweight. (I say technical work because there’s also the interpersonal and inter-organizational work, and I’m not focused on those in this post.)

Let’s think about what’s really meant by “devs fill out self-service security questionnaires…” They’re answering questions about the first two questions in threat modeling: “what are we working on,” and “what can go wrong?” If the answers reach a threshold, then they consider what they’re going to do about it.

And so, they’re threat modeling using a questionnaire. In fact, every engineer (or scrum master) is threat modeling every feature. They’re just using different tools than were in that initial mental model.

And so the question we can ask is not “what do you threat model”, but “how can we best use the time available?” and even add, “…when many threat modeling activities turn out to provide assurance but not discovery of interesting threats?” Before I get there, I want to briefly explain that I’m avoiding the phrase “the highest ROI approach,” because higher ROI activities might involve a high minimum investment, and in this scenario, avoiding that is the price of the seat at the table.

So with that, I’m modeling all questionnaires as the same, but that’s obviously wrong. It’s more useful to characterize the questionnaires. How long do they take? Are they a quick check in, or a tedious waste? How often does each element trigger? What’s the result of those triggers? Are the triggers avoidable with other mechanisms? For example, can you do static analysis grep checkins to find obviously risky variables like SSN, CCN or COVID? Can we do a risk bounty, where the person who points out the highest risk code in a unit gets a thing? Can we make what I’ll call “Big Wall Map” threat models part of scrum planning?

By “Big Wall Map” I mean a way of addressing what we’re working on. There’s a large map of the code on the wall, and scrum feature discussion starts with showing where the changes will take place. Alice might say “My code will change the way we calculate the ads, but still sending the ad request to this module, so there’s no new data flow, and no new type of data crossing the trust boundary.” And she’s done. Billie might have a different change that adds a new data flow, and so she’ll know (or be told) that she needs to do deeper analysis of what can go wrong. This is somewhat similar to a questionnaire, but the compliance check is done by the team and the scrum master. The analysis is dependent on the team. There’s less effort, less output, less evidence. But to the question: “how can we best use the time available?”

Big Wall Map is exceptionally fast in steady state. When you start with Big Wall Map, making of the map is technical debt. Maintaining the map can be expensive. But it has payoff in reducing rework that’s bigger than security. The physical nature of the map is also important in the same way that physical kanban boards are important. They’re visible when people are doing their day jobs, and there’s only so much “good wall space.” What it’s spent on is important. And yes, many people are working from home right now, which makes that less visible.

Back to the goal, what is the goal of the questionnaire? Is it to ensure that nothing slips through the cracks? That there’s enough security analysis of each story? That there’s a record of the analysis so there’s someone to blame? Is it to allocate work by security engineers? Engineering is all about tradeoffs. Crisply defining what we want will help us get there.

Agile folks I work with love to say “if it hurts, do it more.” That’s a great approach for threat modeling, and it’s worth talking through how we can re-factor threat modeling to integrate better. I see a lot of success with developers owning security, Big Wall Maps, and threat modeling every feature in super-lightweight ways. But it takes each company time and work to get to that point. It has to be a cultural journey as well as a technical one.

Image by Andreas Breitling.

3 Comments on "Threat Modeling with Questionnaires"

  1. I don’t know if devs-fill-self-service-security-questionnaires is truly devs doing threat modeling. They may be feeding the requested information without understanding the underlying risk, feeding into a pre-defined risk profile that aims at saving time for the bottleneck security team. Consider “Are you writing in C/C++? (Yes/No)“. The risk is transparent to the developer. They are not exposed to what might go wrong. The security team will channel a Yes answer into “take a deeper look”. I feel that this leads us to invariants – “previous threat models and actual results showed us that C/C++ is problematic because of memory errors. Do you mind removing this whole class of issues and saving us a ton of SCA money by writing your stuff in Java?“. So the questionnaire now is not looking for risk, it is looking for invariants. Move that to cloud work. There are only some ways that you can securely and efficiently do work on GC/AWS/Azure. Make those into playbooks, invariants, and now measure in threat models any deviation from them. The questionnaire then becomes much less “what could go wrong” and more “how are you introducing variations into the thing I told you is right to do”. The questionnaire becomes more focused on the real meat of the question. That’s where I feel the next stage for “Threat Model Every Story” is heading. But I guess I digress by a mile and a half.
    The ideal of the Big Wall Map is awesome, carrying as costs all the things you pointed out (and the price of not paying that cost steadily is a wrong Big Wall Map, which may be worse than no Big Wall Map at all). The payoff is also beyond security, as you point out, but how do we make the BWM reflect all the information needed to make correct evaluations of the state of the system as it changes? Is the BWM a DFD? Is there value to keep the BWM on the wall or do we need a digital representation of it? Who owns the BWM and can make changes to it? Do changes happen at design time or when the feature is accepted? Is even “features” a unit for BWM ?

    I think your analysis of the mental model is the meat of the question, but I’d say the way to move away is to focus on what is the final expected mental model by both the devs and the security team (if any). If Billie is told about the need for a deeper analysis, will Billie ever get the mindset needed to not be told? What can we give Billie outside the specific “do deeper analysis” to help create that “I am defining a new workflow, I better do some deeper analysis” mindset?

  2. (Copied from your LinkedIn post here:

    Thanks for taking the time to write this up! Really interesting read. A few thoughts:

    I like the nuance called out here:

    > ‘I’m avoiding the phrase “the highest ROI approach,” because higher ROI activities might involve a high minimum investment, and in this scenario, avoiding that is the price of the seat at the table.’

    The “Big Wall Map” is a cool idea! In your view, who creates it, when, and how?

    * Probably devs, as they have the most context, but then we’re giving them more work.
    * Seems like a large upfront and recurring time investment.
    * Have you seen any small or medium sized businesses successfully use this approach? I could see larger/more established organizations view the trade-off as worth it.

    Love this part:

    > “What is the goal of the questionnaire? Is it to ensure that nothing slips through the cracks? That there’s enough security analysis of each story? … Is it to allocate work by security engineers? Engineering is all about trade-offs. Crisply defining what we want will help us get there.”

    Lastly, totally agree re: having devs own security and threat modeling every feature in a lightweight way.

    I actually also called that out as an option in my slides (slide #155):

    1. Thanks! Responding to the part about BWM.

      I want to separate who does the work from what they do. At an org with strong technical program managers, they could own such a map. If the PMs are more project management, then they can’t.

      Roughly all orgs start without a map, and creating it is technical debt (security debt) that must be scheduled like any other work. I’ve seen that work shift rapidly from ‘why bother’ to ‘yes, we should’ after 6-12 weeks of standups without a map.

      How much ongoing work is involved is really dependant on the rate of architectural change, which itself changes over time. If the practice is to draw on the map during standups as features change the map, then you need less technical knowledge to make a pretty version of the map.

      I have seen this at smaller orgs, but most of my practice is helping larger orgs so I have less to say there.

Comments are closed.