I’m happy to say that some new research by Jay Jacobs, Wade Baker, and myself is now available, thanks to the Global Cyber Alliance.
They asked us to look at the value of DNS security, such as when your DNS provider uses threat intel to block malicious sites. It’s surprising how effective it is for a tool that’s so easy to deploy. (Just point to a DNS server like 9.9.9.9).
The report is available from GCA’s site: Learn About How DNS Security Can Mitigate One-Third of Cyber Incidents
The House Oversight Committee has released a scathing report on Equifax.
Through the investigation, the Committee reviewed over 122,000 pages of documents, conducted transcribed interviews with three former Equifax employees directly involved with IT, and met with numerous current and former Equifax employees, in addition to Mandiant, the forensic firm hired to conduct an investigation of the breach.
I haven’t had time to review the report in detail, but I don’t think it answers questions I had reading the GAO report. Four of their give key findings are about what happened before the breach, but the fifth, “unprepared to support affected consumers,” goes to a point I’ve made consistently over nearly a dozen years: “
It’s Not The Crime, It’s The Coverup or the Chaos.”
I have regularly asked why we don’t know more about the Equifax breach, including in comments in “That Was Close! Reward Reporting of Cybersecurity ‘Near Misses’.” These questions are not intended to attack Equifax. Rather, we can use their breach as a mirror to reflect, and ask questions about how defenses work, and learn things we can bring to our own systems.
Ian Melvin was kind enough to point out a GAO report, “Actions Taken by Equifax and Federal Agencies in Response to the 2017 Breach.” As you’d expect of a GAO report, it is level headed and provides a set of facts.
However, I still have lots of questions. Some very interesting details start on page 11:
Equifax officials added that, after gaining the ability to issue system-level commands on the online dispute portal that was originally compromised, the attackers issued queries to other databases to search for sensitive data. This search led to a data repository containing PII, as well as unencrypted usernames and passwords that could provide the attackers access to several other Equifax databases. According to Equifax’s interim Chief Security Officer, the attackers were able to leverage these credentials to expand their access beyond the 3 databases associated with the online dispute portal, to include an additional 48 unrelated
databases.The use of encryption allowed the attackers to blend in their malicious actions with regular activity on the Equifax network and, thus, secretly maintain a presence on that network as they launched further attacks without being detected by Equifax’s scanning software. (Editor’s note: I’ve inverted the order of the paragraphs from the source.)
So my questions include:
Specifically, while Equifax had installed a device to inspect network traffic for evidence of malicious activity, a misconfiguration allowed encrypted traffic to pass through the network without being inspected. According to Equifax officials, the misconfiguration was due to an expired digital certificate. The certificate had expired about 10 months before the breach occurred, meaning that encrypted traffic was not being inspected throughout that period.
Would your organization notice if one of hundreds or dozens of IDSs shut up for a week, or if one ruleset stopped firing?
More published incident reports will help us get smarter, and provide better answers to the questions that CEOs and boards are asking: could this happen to us? With this report we an answer that better, but still not well.
Well, Richard Smith has “resigned” from Equifax.
The CEO being fired is a rare outcome of a breach, and so I want to discuss what’s going on and put it into context, which includes the failures at DHS, and Deloitte breach. Also, I aim to follow the advice to praise specifically and criticize in general, and break that pattern here because we can learn so much from the specifics of the cases, and in so learning, do better.
Smith was not fired because of the breach. Breaches happen. Executives know this. Boards know this. The breach is outside of their control. Smith was fired because of the post-breach chaos. Systems that didn’t work. Tweeting links to a scam site for two weeks. PINS that were recoverable. Weeks of systems saying “you may have been a victim.” Headlines like “Why the Equifax Breach Stings So Bad” in the NYTimes. Smith was fired in part because of the post-breach chaos, which was something he was supposed to control.
But it wasn’t just the chaos. It was that Equifax displayed so much self-centeredness after the breach. They had the chutzpah to offer up their own product as a remedy. And that self-dealing comes from seeing itself as a victim. From failing to understand how the breach will be seen in the rest of the world. And that’s a very similar motive to the one that leads to coverups.
In The New School Andrew and I discussed how fear of firing was one reason that companies don’t disclose breaches. We also discussed how, once you agree that “security issues” are things which should remain secret or shared with a small group, you can spend all your energy on rules for information sharing, and have no energy left for actual information sharing.
And I think that’s the root cause of “U.S. Tells 21 States That Hackers Targeted Their Voting Systems” a full year after finding out:
The notification came roughly a year after officials with the United States Department of Homeland Security first said states were targeted by hacking efforts possibly connected to Russia.
A year.
A year.
A year after states were first targeted. A year in which “Obama personally warned Mark Zuckerberg to take the threats of fake news ‘seriously.’” (Of course, the two issues may not have been provably linkable at the time.) But. A year.
I do not know what the people responsible for getting that message to the states were doing during that time, but we have every reason to believe that it probably had to do with (and here, I am using not my sarcastic font, but my scornful one) “rules of engagement,” “traffic light protocols,” “sources and methods” and other things which are at odds with addressing the issue. (End scornful font.) I understand the need for these things. I understand protecting sources is a key role of an intelligence service which wants to recruit more sources. And I also believe that there’s a time to risk those things. Or we might end up with a President who has more harsh words for Australia than the Philippines. More time for Russia than Germany.
In part, we have such a President because we value secrecy over disclosure. We accept these delays and view them as reasonable. Of course, the election didn’t turn entirely on these issues, but on our electoral college system, which I discussed at some length, including ways to fix it.
All of which brings me to the Deloitte breach, “Deloitte hit by cyber-attack revealing clients’ secret emails.” Deloitte, along with the others who make up the big four audit firms, gets access to its clients deepest secrets, and so you might expect that the response to the breach would be similar levels of outrage. And I suspect a lot of partners are making a lot of hat-in-hand visits to boardrooms, and contritely trying to answer questions like “what the flock were you people doing?” and “why the flock weren’t we told?” I expect that there’s going to be some very small bonuses this year. But, unlike our relationship with Equifax, boards do not feel powerless in relation to their auditors. They can pick and swap. Boards do not feel that the system is opaque and unfair. (They sometimes feel that the rules are unfair, but that’s a different failing.) The extended reporting time will likely be attributed to the deep analysis that Deloitte did so it could bring facts to its customers, and that might even be reasonable. After all, a breach is tolerable; chaos afterwards may not be.
The two biggest predictors of public outrage are chaos and coverups. No, that’s not quite right. The biggest causes are chaos and coverups. (Those intersect poorly with data brokerages, but are not limited to them.) And both are avoidable.
So what should you do to avoid them? There’s important work in preparing for a breach, and in preventing one.
I’d love to hear what other steps you think organizations often miss out on.
[Thursday, September 21th is the latest of 5 updates.]
When I wrote “The Breach Response Market Is Broken,” I didn’t expect one of the players to validate everything I had to say. What I said was that the very act of firms contracting with breach response services inhibit the creation of a market for breach response, and the FTC should require them to give vouchers to consumers.
Vice Motherboard is reporting that “Firm Hired to Monitor Data Breaches Is Hacked, 143 Million Social Security Numbers Stolen.”
It’s not clear what database was accessed. On their website, Equifax says “No Evidence of Unauthorized Access to Core Consumer or Commercial Credit Reporting Databases” and “Company to Offer Free Identity Theft Protection and Credit File Monitoring to All U.S. Consumers.”
But here’s the thing; I don’t trust Equifax to protect data that … they just failed to protect. I want protection from an independent firm.
Equifax’s self-dealing in providing breach response services is unfair. No rational, well-informed consumer would select Equifax’s service in this situation. Equifax’s offering of credit file monitoring to all US consumers is also an unfair trade practice, which undercuts innovation, and limits the ability of new entrants to deliver effective services.
The FTC should require Equifax to send a voucher to each impacted individual which can be used to purchase any identity theft protection service on the market as of August, 2017.
Update 1, later that day:
Update 2, Sept 9:
Update 3, Sept 12:
Update 4, September 16:
Update 5, September 21:
There’s a great “long read” at CIO, “6 Software Development Lessons From Healthcare.gov’s Failed Launch.” It opens:
This article tries to go further than the typical coverage of Healthcare.gov. The amazing thing about this story isn’t the failure. That was fairly obvious. No, the strange thing is the manner in which often conflicting information is coming out. Writing this piece requires some archeology: Going over facts and looking for inconsistencies to assemble the best information about what’s happened and pinpoint six lessons we might learn from it.
There’s a lot there, and I liked it even before lesson 6 (“Threat Modeling Matters”). Open analysis is generally better.
There’s a question of why this has to be done by someone like Matthew Heusser. No disrespect is intended, but why isn’t Healthcare.gov performing these analyses and sharing them? Part of the problem is that we live in an “outrage world” where it’s easier to point fingers and giggle in 140 characters and hurt people’s lives or careers than it is to make a positive contribution.
It would be great to see project analyses and attempts to learn from more projects that go sideways. But it would also be great to see these for security failures. As I asked in “What Happened At OPM,” we have these major hacks, and we learn nothing at all from them. (Or worse, we learn bad lessons, such as “don’t go looking for breaches.”)
The definition of insanity is doing the same thing over and over and hoping for different results. (Which may includes asking the same question or writing the same blog post over and over, which is why I’m starting a company to improve security effectiveness.)
I want to discuss some elements of the OPM breach and what we know and what we don’t. Before I do, I want to acknowledge the tremendous and justified distress that those who’ve filled out the SF-86 form are experiencing. I also want to acknowledge the tremendous concern that those who employ those with clearances must be feeling. The form is designed as an (inverted) roadmap to suborning people, and now all that data is in the hands of a foreign intelligence service.
The National Journal published A Timeline of Government Data Breaches:
I asked after the root cause, and Rich Bejtlich responded “The root cause is a focus on locking doors and windows while intruders are still in the house” with a pointer to his “Continuous Diagnostic Monitoring Does Not Detect Hackers.”
And while I agree with Richard’s point in that post, I don’t think that’s the root cause. When I think about root cause, I think about approaches like Five Whys or Ishikawa. If we apply this sort of approach then we can ask, “Why were foreigners able to download the OPM database?” There are numerous paths that we might take, for example:
Alternately, we might go down a variety of paths based on the Inspector General Report. We might consider Richard’s point:
But we can go down alternate paths:
What I’d really like to see are specific technical facts laid out. (Or heck, if the facts are unknowable because logs rotated or attackers deleted them, or we don’t even know why we can’t know, let’s talk about that, and learn from it.)
OPM and Katherine Archuleta have already been penalized. Let’s learn things beyond dates. Let’s put the facts out there, or, as I quoted in my last post we “should declare the causes which impel them to the separation”, or “let Facts be submitted to a candid world.” Once we have facts about the causes, we can perform deeper root cause analysis.
I don’t think that the OIG report contains those causes. Each of those audit failings might play one of several roles. The failing might have been causal, and fixing it would have stopped the attack. The failing might have been casual and the attacker would have worked around it. The failing might be irrelevant (for example, I’ve rarely seen an authorization to operate prevent an attack, unless you fold it up very small and glue it into a USB port). The failings might even have been distracting, taking attention away from other work that might have prevented the attack.
A collection of public facts would enable us to have a discussion about those possibilities. We could have a meta-conversation about those categorizations of failings, and if there’s other ones which make more sense.
Alternately, we can keep going the way we’re going.
So. What happened at OPM?
Governor Brown of California has signed a strengthened breach notification bill, which amends Sections 1798.29 and 1798.82 of the California Civil Code in important ways. Previous versions had been repeatedly vetoed by Arnold Schwarzenegger.
As described[.DOC] by its sponsor’s office, this law:
Establishes standard, core content — such as the type of information breached, time of breach, and toll-free telephone numbers and addresses of the major credit reporting agencies — for security breach notices in California; Requires public agencies, businesses, and persons subject to California’s security breach notification law, if more than 500 California residents are affected by a single breach, to send an electronic copy of the breach notification to the Attorney General; and, Requires public agencies, businesses and persons subject to California’s security breach notification law, if they are utilizing the substitute notice provisions in current law, to also provide that notification to the Office of Information Security or the Office of Privacy Protection, as applicable.
This makes California the fifteenth (!) state with a central notification provision on the books, the others being: Hawaii, Iowa, Maryland, Massachusetts, Minnesota, New Hampshire, New York, North Carolina, Oregon, Vermont, Virginia, West Virginia, Wisconsin, and Wyoming. Puerto Rico also has such a requirement. Ibid.
I’m looking forward to the resulting information, and I hope California’s Attorney General has the good sense to post all received notification letters. This will undoubtedly be easier for the state than dealing with the inevitable FOIA requests, and serves the public interest by increasing transparency.
JC Penney, Wet Seal: Gonzalez Mystery Merchants
JCPenney and Wet Seal were both officially added to the list of retail victims of Albert Gonzalez on Friday (March 26) when U.S. District Court Judge Douglas P. Woodlock refused to continue their cloak of secrecy and removed the seal from their names. StorefrontBacktalk had reported last August that $17 billion JCPenney chain was one of Gonzalez’s victims, even though JCPenney’s media representatives were denying it.
and
[The judge said] both retailers should have announced their involvement from the start, that consumers had the right to know. He said he would not provide the companies “insulation from transparency.” The judge stressed that the companies were seeking privacy as though they were individual victims, which he said was like “hermaphroditing a business corporation.”
Wired picked up the story and wrote:
It’s a bit jarring to see a lucid pro-transparency, pro-security argument from a federal prosecutor. For years, law enforcement has had an informal policy of protecting companies from the public relations consequences of their poor security — a kind of omerta among intruders, the companies they hack and the feds, where only the public is left in the dark. To be sure, it’s never been set in stone, and not all feds have played ball. But it’s a common practice, and it corrodes accountability.
Precision blogging gets the scoop:
You’re probably talking about this terrible security disaster already: the largest database leak ever. Arweena, a spokes-elf for Santa Claus, admitted a few hours ago that the database posted at WikiLeaks yesterday is indeed the comprehensive 2009 list of which kids have been naughty, and which were nice. The source of the leak is unclear. It may have come from a renegade reindeer, or it could be the work of a clever programmer in the Ukraine. Either way, it’s a terrible black eye for Santa. Arweena promised that in the future, access to this database would be restricted on a “need to know” basis. And you know who that means!
Let’s see if customers really change their behavior. I know which way I’m betting.