To what Bellovin says, I would add that 15 years ago, Steve Beattie, Crispin Cowan and I did some math for Timing the Application of Security Patches for Optimal Uptime, and estimated that likelihood of attack starts to exceed likelihood of damage from the patch at around 10 days. To my knowledge, no one has updated the dataset or re-run the numbers, but I would expect that improvements in test automation and improvement in attack frameworks make that closer to patch release, not further from it. My experience is that many organizations with dependencies on older technology also have not invested in test automation that enables even fast ‘smoke testing’ of their systems. Such test rigs allow you to quickly start the clock that Steve hypothesizes.
This is an thought-provoking story:
And perhaps most unusual, the FBI recently obtained a single warrant in Alaska to hack the computers of thousands of victims in a bid to free them from the global botnet, Kelihos.
On April 5, Deborah M. Smith, chief magistrate judge of the US District Court in Alaska, greenlighted this first use of a controversial court order. Critics have since likened it to a license for mass hacking. (“FBI allays some critics with first use of new mass-hacking warrant,” Aliya Sternstein, Ars Technica)
One of the issues in handling malware at scale is that the law prohibits unauthorized access to computers. And that’s not a bad principle, but it certainly makes it challenging to go and clean up infections at scale.
So the FBI getting a warrant to authorize that may be an interesting approach, with many cautions about the very real history of politicized spying, of COINTEL, of keeping the use of 0days secret. But I don’t want to focus on those here. What I want to focus on is what did a judge actually authorize in this case. Is this a warrant for mass hacking? It doesn’t appear to be, but it does raise issues of what’s authorized, how those things were presented to the judge, and points to future questions about what authorization might include.
So what’s authorized?
The application for a warrant is fairly long, at 34 pages, much of it explaining why the particular defendant is worthy of a search, and the “time and manner of execution of search” starts on page 30.
What the FBI apparently gets to do is to operate a set of supernodes for the Kelihos botnet, and “The FBI’s communications, however, will not contain any commands, nor will they contain IP addresses of any of the infected computers. Instead, the FBI replies will contain the IP and routing information for the FBI’s ‘sinkhole’ server.”
What questions does that raise?
A first technical point is that for the FBI’s replies to reach those infected computers, there must be packets sent over the internet. For those packets to reach the infected computer’s, they need addressing, in the form of an IP address. Now you can argue that those IP addresses of infected computers are in the headers, not the content of the packets. The nuance of content versus headers is important in some laws. In fact, the warrant para 66 explicitly states that the FBI will gather that data, and then provide it to ISPs, who they hope will notify end users. (I’ve written about that experience in “The Worst User Experience In Computer Security?.”)
Another technical point is that the warrant says “The FBI with the assistance of private parties…” It’s not clear to me what constraints might apply to those parties. Can they record netflow or packet captures? (That might be helpful in demonstrating exactly what they did later, and also create a privacy conundrum which the FBI takes apparent pains to avoid.) What happens if an unrelated party captures that data? For example, let’s say one of the parties chooses to operate their sinkhole in an AWS node. I suspect AWS tracks netflows. A warrant to obtain that data might seem pretty innocent to a judge.
The idea that the FBI will not send any commands is, on the surface, a fine restriction, but it eliminates many possibilities for cleaning up. What could we imagine in the future?
For example, a command that could be useful would be to remove Kelihos from startup items? How about remove C:\Program Files\Kelihos.exe? Removing files from my computer without permission is pretty clearly a form of unauthorized access. It’s a form that’s probably in the interests of the vast majority of the infected. We might want to allow a well-intentioned party to do so.
But what if the commands fail? When Sony built a tool to remove the rootkit their DRM installed, the cleanup tool opened a big security hole. It’s hard to write good code. It’s very hard to write code that’s free of side effects or security issues.
What if there’s disagreement over what fits within the definition of well-intentioned? What if someone wants to remove duplicate files to help me save disk space? What if they want to remove blasphemous or otherwise illegal files?
Threat modeling internet-enabled things is similar to threat modeling other computers, with a few special tensions that come up over and over again. You can start threat modeling IoT with the four question framework:
- What are you building?
- What can go wrong?
- What are you going to do about it?
- Did we do a good job?
But there are specifics to IoT, and those specifics influence how you think about each of those questions. I’m helping a number of companies who are shipping devices, and I would love to fully agree that “consumers shouldn’t have to care about the device’s security model. It should just be secure. End of story.” I agree with Don Bailey on the sentiment, but frequently the tensions between requirements mean that what’s secure is not obvious, or that “security” conflicts with “security.” (I model requirements as part of ‘what are you building.’)
When I train people to threat model, I use this diagram to talk about the interaction between threats, mitigations, and requirements:
The interaction has a flavor in working with internet-enabled things, and that interaction changes from device to device. There are some important commonalities.
When looking at what you’re building, IoT devices typically lack sophisticated input devices like keyboards or even buttons, and sometimes their local output is a single LED. One solution is to put a web server on the device listening, and to pay for a sticker with a unique admin password, which then drives customer support costs. Another solution is to have the device not listen but to reach out to your cloud service, and let customers register their devices to their cloud account. This has security, privacy, and COGS
downsides tradeoffs. [Update: I said downsides, but it’s more a different set of attack vectors become relevant in security. COGS is an ongoing commitment to operations; privacy is dependent on what’s sent or stored.]
When asking what can go wrong, your answers might include “a dependency has a vulnerability,” or “an attacker installs their own software.” This is an example of security being in tension with itself is the ability to patch your device yourself. If I want to be able to recompile the code for my device, or put a safe version of zlib on there, I ought to be able to do so. Except if I can update the device, so can attackers building a botnet, and 99.n% of typical consumers for a smart lightbulb are not going to patch themselves. So we get companies requiring signed updates. Then we get to the reality that most consumer devices last longer than most silicon valley companies. So we want to see a plan to release the key if the company is unable to deliver updates. And that plan runs into the realities of bankruptcy law, which is that that signing key is an asset, and its hard to value, and bankruptcy trustees are unlikely to give away your assets. There’s a decent pattern (allegedly from the world of GPU overclocking), which is that you can intentionally make your device patchable by downloading special software and moving a jumper. This requires a case that can be opened and reclosed, and a jumper or other DFU hardware input, and can be tricky on inexpensive or margin-strained devices.
That COGS (cost of goods sold) downside is not restricted to security, but has real security implications, which brings us to the question of what are you going to do about it. Consumers are not used to subscribing to their stoves, nor are farmers used to subscribing to their tractors. Generally, both have better things to do with their time than to understand the new models. But without subscription revenue, it’s very hard to make a case for ongoing security maintenance. And so security comes into conflict with consumer mental models of purchase.
In the IoT world, the question of did we do a good job becomes have we done a good enough job? Companies believe that there is a first mover advantage, and this ties to points that Ross Anderson made long ago about the tension between security and economics. Good threat modeling helps companies get to those answers faster. Sharing the tensions help us understand what the tradeoffs look like, and with those tensions, organizations can better define their requirements and get to a consistent level of security faster.
I would love to hear your experiences about other issues unique to threat modeling IoT, or where issues weigh differently because of the IoT nature of a system!
(Incidentally, this came from a question on Twitter; threading on Twitter is now worse than it was in 2010 or 2013, and since I have largely abandoned the platform, I can’t figure out who’s responding to what. A few good points I see include:
That’s the subtitle of a new paper by Cormac Herley and Paul van Oorschot, “SoK: Science, Security, and the Elusive Goal of Security as a Scientific Pursuit,” forthcoming in IEEE Security & Privacy.
The past ten years has seen increasing calls to make security research more “scientific”. On the surface, most agree that this is desirable, given universal recognition of “science” as a positive force. However, we find that there is little clarity on what “scientific” means in the context of computer security research, or consensus on what a “Science of Security” should look like. We selectively review work in the history and philosophy of science and more recent work under the label “Science of Security”. We explore what has been done under the theme of relating science and security, put this in context with historical science, and offer observations and insights we hope may motivate further exploration and guidance. Among our findings are that practices on which the rest of science has reached consensus appear little used or recognized in security, and a pattern of methodological errors continues unaddressed.
There’s a very interesting paper on the Cyber Grand Challenge by team Shellphish. Lots of details about the grand challenge itself, how they designed their software, how they approached the scoring algorithm, and what happened in the room.
There’s lots of good details, but perhaps my favorite is:
How would a team that did *nothing* do? That is, if a team connected and then ceased to play, would they fare better or worse than the other players? We ran a similar analysis to the “Never patch” strategy previously (i.e., we counted a CS as exploited for all rounds after its first exploitation against any teams), but this time removed any POV-provided points. In the CFE, this “Team NOP” would have scored 255,678 points, barely *beating* Shellphish and placing 3rd in the CGC.
The reason I like this is that scoring systems are hard. Really, really hard. I know that DARPA spent substantial time and energy on the scoring system, and this outcome happened anyway. We should not judge either DARPA or the contest on that basis, because it was hard to see that that would happen ahead of time: it’s a coincidence of the scores teams actually achieved.
When I think about how to threat model well, one of the elements that is most important is how much people need to keep in their heads, the cognitive load if you will.
In reading Charlie Stross’s blog post, “Writer, Interrupted” this paragraph really jumped out at me:
One thing that coding and writing fiction have in common is that both tasks require the participant to hold huge amounts of information in their head, in working memory. In the case of the programmer, they may be tracing a variable or function call through the context of a project distributed across many source files, and simultaneously maintaining awareness of whatever complex APIs the object of their attention is interacting with. In the case of the author, they may be holding a substantial chunk of the plot of a novel (or worse, an entire series) in their head, along with a model of the mental state of the character they’re focussing on, and a list of secondary protagonists, while attempting to ensure that the individual sentence they’re currently crafting is consistent with the rest of the body of work.
One of the reasons that I’m fond of diagrams is that they allow the threat modelers to migrate information out of their heads into a diagram, making room for thinking about threats.
Lately, I’ve been thinking a lot about threat modeling tools, including some pretty interesting tools for automated discovery of existing architecture from code. That’s pretty neat, and it dramatically cuts the cost of getting started. Reducing effort, or cost, is inherently good. Sometimes, the reduction in effort is an unalloyed good, that is, any tradeoffs are so dwarfed by benefits as to be unarguable. Sometimes, you lose things that might be worth keeping, either as a hobby like knitting or in the careful chef preparing a fine meal.
I think a lot about where drawing diagrams on a whiteboard falls. It has a cost, and that cost can be high. “Assemble a team of architect, developer, test lead, business analyst, operations and networking” reads one bit of advice. That’s a lot of people for a cross-functional meeting.
That meeting can be a great way to find disconnects in what people conceive of building. And there’s a difference between drawing a diagram and being handed a diagram. I want to draw that out a little bit and ask for your help in understanding the tradeoffs and when they might and might not be appropriate. (Gary McGraw is fond of saying that getting these people in a room and letting them argue is the most important step in “architectural risk analysis.” I think it’s tremendously valuable, and having structures, tools and methods to help them avoid ratholes and path dependency is a big win.)
So what are the advantages and disadvantages of each?
- Collaboration. Walking to the whiteboard and picking up a marker is far less intrusive than taking someone’s computer, or starting to edit a document in a shared tool.
- Ease of use. A whiteboard is still easier than just about any other drawing tool.
- Discovery of different perspective/belief. This is a little subtle. If I’m handed a diagram, I’m less likely to object. An objection may contain a critique of someone else’s work, it may be a conflict. As something is being drawn on a whiteboard, it seems easier to say “what about the debug interface?” (This ties back to Gary McGraw’s point.)
- Storytelling. It is easier to tell a story standing next to a whiteboard than any tech I’ve used. A large whiteboard diagram is easy to point at. You’re not blocking the projector. You can easily edit as you’re talking.
- Messy writing/what does that mean? We’ve all been there? Someone writes something in shorthand as a conversation is happening, and either you can’t read it or you can’t understand what was meant. Structured systems encourage writing a few more words, making things more tedious for everyone around.
- Automatic analysis. Tools like the Microsoft Threat Modeling tool can give you a baseline set of threats to which you add detail. Structure is a tremendous aid to getting things done, and in threat modeling, it helps in answering “what could go wrong?”
- Authority/decidedness/fixedness. This is the other side of the discovery coin. Sometimes, there are architectural answers, and those answers are reasonably fixed. For example, hardware accesses are mediated by the kernel, and filesystem and network are abstracted there. (More recent kernels offer filesystems in userland, but that change was discussed in detail.) Similarly, I’ve seen large, complex systems with overall architecture diagrams, and a change to these diagrams had to be discussed and approved in advance. If this is the case, then a fixed diagram, printed poster size and affixed to walls, can also be used in threat modeling meetings as a context diagram. No need to re-draw it as a DFD.
- Photographs of whiteboards are hard to archive and search without further processing.
- Photographs of whiteboards may imply that ‘this isn’t very important.” If you have a really strong culture of “just barely good enough” than this might not be the case, but if other documents are more structured or cared for, then photos of a whiteboard may carry a message.
- Threat modeling only late. If you’re going to get architecture from code, then you may not think about it until the code is written. If you weren’t going to threat model anyway, then this is a win, but if there was a reasonable chance you were going to do the architectural analysis while there was a chance to change the architecture, software tools may take that away.
(Of course, there are apps that help you take images from a whiteboard and improve them, for example, Best iOS OCR Scanning Apps, which I’m ignoring for purposes of teasing things out a bit. Operationally, probably worth digging into.)
I’d love your thoughts: are there other advantages or disadvantages of a whiteboard or software?
At the RMS blog, we learn they are “Launching a New Journal for Terrorism and Cyber Insurance:”
Natural hazard science is commonly studied at college, and to some level in the insurance industry’s further education and training courses. But this is not the case with terrorism risk. Even if insurance professionals learn about terrorism in the course of their daily business, as they move into other positions, their successors may begin with hardly any technical familiarity with terrorism risk. It is not surprising therefore that, even fifteen years after 9/11, knowledge and understanding of terrorism insurance risk modeling across the industry is still relatively low.
There is no shortage of literature on terrorism, but much has a qualitative geopolitical and international relations focus, and little is directly relevant to terrorism insurance underwriting or risk management.
This is particularly exciting as Gordon Woo was recommended to me as the person to read on insurance math in new fields. His Calculating Catastrophe is comprehensive and deep.
It will be interesting to see who they bring aboard to complement the very strong terrorism risk team on the cyber side.
No, seriously. Articles like “Microsoft Secure Boot key debacle causes security panic” and “Bungling Microsoft singlehandedly proves that golden backdoor keys are a terrible idea” draw on words in an advisory to say that this is all about golden keys and secure boot. This post is not intended to attack anyone; researchers, journalists or Microsoft, but to address a rather inflammatory claim that’s being repeated.
Based on my read of a advisory copy (which I made because I cannot read words on an animated background (yes, I’m a grumpy old man (who uses too many parentheticals (especially when I’m sick)))), this is a nice discovery of an authorization failure.
What they found is:
The “supplemental” policy contains new elements, for the merging conditions. These conditions are (well, at one time) unchecked by bootmgr when loading a legacy policy. And bootmgr of win10 v1511 and earlier certainly doesn’t know about them. To those bootmgrs, it has just loaded in a perfectly valid, signed policy. The “supplemental” policy does NOT contain a DeviceID. And, because they were meant to be merged into a base policy, they don’t contain any BCD rules either, which means that if they are loaded, you can enable testsigning.
That’s a fine discovery and a nice vuln. There are ways Microsoft might have designed this better, I’m going to leave those for another day.
Where the post goes off the rails, in my view, is this:
About the FBI: are you reading this? If you are, then this is a perfect real world example about why your idea of backdooring cryptosystems with a “secure golden key” is very bad! Smarter people than me have been telling this to you for so long, it seems you have your fingers in your ears. You seriously don’t understand still? Microsoft implemented a “secure golden key” system. And the golden keys got released from MS own stupidity. Now, what happens if you tell everyone to make a “secure golden key” system?  (Bracketed numbers added – Adam)
So, , no they did not.  No it didn’t.  Even a stopped clock …
You could design a system in which there’s a master key, and accidentally release that key. Based on the advisory, Microsoft has not done that. (I have not talked to anyone at MS about this issue; I might have talked to people about the overall design, but don’t recall having done so.) What this is is an authorization system with a design flaw. As far as I can tell, no keys have been released.
Look, there are excellent reasons to not design a “golden key” system. I talked about them at a fundamental engineering level in my threat modeling book, and posted the excerpt in “Threat Modeling Crypto Back Doors.”
The typical way the phrase “golden key” is used (albiet fuzzily) is that there is a golden key which unlocks communications. That is a bad idea. This is not that, and we as engineers or advocates should not undercut our position on that bad idea by referring to this research as if it really impacts on that “debate.”
C-3PO: Sir, the possibility of successfully navigating an asteroid field is approximately 3,720 to 1.
Han Solo: Never tell me the odds.
I was planning to start this with a C-3PO quote, and then move to a discussion of risk and risk taking. But I had forgotten just how rich a vein George Lucas tapped into with 3PO’s lines in The Empire Strikes Back. So I’m going to talk about his performance prior to being encouraged to find a non-front-line role with the Rebellion.
In case you need a refresher on the plot, having just about run out of options, Han Solo decides to take the known, high risk of flying into an asteroid field. When he does, 3PO interrupts to provide absolutely useless information. There’s nothing about how to mitigate the risk (except surrendering to the Empire). There’s nothing about alternatives. Then 3PO pipes up to inform people that he was previously wrong:
C-3PO: Artoo says that the chances of survival are 725 to 1. Actually Artoo has been known to make mistakes… from time to time… Oh dear…
I have to ask: How useless is that? “My first estimate was off by a factor of 5, you should trust this new one?”
C-3PO: I really don’t see how that is going to help! Surrender is a perfectly acceptable alternative in extreme circumstances! The Empire may be gracious enough to… [Han signals to Leia, who shuts 3PO down.]
Most of the time, being shut down in a meeting isn’t this extreme. But there’s a point in a discussion, especially in high-pressure situations, where the best contribution is silence. There’s a point at which talking about the wrong thing at the wrong time can cost credibility that you’d need later. And while the echo in the dialogue is for comic effect, the response certainly contains a lesson for us all:
C-3PO: The odds of successfully surviving an attack on an Imperial Star Destroyer are approximately…Leia: Shut up!
And the eventual outcome:
C-3PO: Sir, If I may venture an opinion…Han Solo: I’m not really interested in your opinion 3PO.
Does C-3PO resemble any CSOs you know? Any meetings you’ve been in? Successful business people are excellent at thinking about risk. Everything from launching a business to hiring employees to launching a new product line involves risk tradeoffs. The good business people either balance those risks well, or they transfer them away in some way (ideally, ethically). What they don’t want or need is some squeaky-voiced robot telling them that they don’t understand risk.
So don’t be C-3PO. Provide useful input at useful times, with useful options.
Originally appeared in Dark Reading, “Security Lessons from C-3PO, Former CSO of the Millennium Falcon,” as part of a series I’m doing there, “security lessons from..”. So far in the series, lessons from: my car mechanic, my doctor, The Gluten Lie, and my stock broker.
Have a survival kit: ricola, Purell, gatorade, advil and antacids can be brought or bought on site.
Favorite talk (not by me): I look forward to Sounil Yu’s talk on “Understanding the Security Vendor Landscape Using the Cyber Defense Matrix.” I’ve seen an earlier version of this, and like the model he’s building a great deal.
Favorite talk I’m giving: “Securing the ‘Weakest Link’.”
A lot of guides, like this one, are not very comprehensive or strategic. John Masserini’s A CISO’s Guide to RSA Conference 2016 is a very solid overview if you’re new, or not getting good value from a conference.
While you’re there, keep notes for a trip report. Sending a trip report helps you remember what happened, helps your boss understand why they spent the money, and helps justify your next trip. I like trip reports that start with a summary, go directly to action items, then a a list of planned meetings and notes on them, followed by detailed and organized notes.
Also while you’re there, remember it’s infosec, and drama is common. Remember the drama triangle and how to avoid it.