Exploitation of Exchange Server spreads rapidly across the globe. The US mulls its response to Russia over the SolarWinds compromise (and to China over Exchange Server hacks).

Dave Bittner: Threat actors rush to exploit Exchange Server vulnerabilities before victims get around to patching. It's like a worldwide fire sale. Rick Howard digs into third-party platforms and cloud security. Robert M. Lee from Dragos shares insights on the recent Florida water plant event. The U.S. mulls some form of retaliation against Russia for the SolarWinds supply chain campaign, and it will also need to consider how to respond to China's operations against Exchange Server. And another Chinese threat actor may have been exploiting SolarWinds late last year.

Dave Bittner: From the CyberWire studios at DataTribe, I'm Dave Bittner with your CyberWire summary for Monday, March 8, 2021. 

Dave Bittner: Chinese threat actors' exploitation of Microsoft Exchange Server zero days has proven about as extensive and damaging as early fears held it to be. Bloomberg sums up current views of the incident, saying that it's morphing into a global cybersecurity crisis with exploitation racing against patching and remediation. KrebsOnSecurity put the total number of US organizations affected by exploitation of the Exchange Server vulnerabilities at about 30,000. The Washington Post reports that the count of victims has already exceeded the number of targets affected by the SolarWinds compromise, nor has the incident been confined to U.S. targets. The European Banking Authority, for one, yesterday disclosed that it, too, had been affected and that it has taken its email systems offline as an initial response. 

Dave Bittner: Not all such exploitation is the work of Hafnium, the Chinese-affiliated threat actor Microsoft identified last week as being behind the campaign. In a Friday update to the relevant security advisory, Redmond wrote, quote, "Microsoft continues to see increased use of these vulnerabilities and attacks targeting unpatched systems by multiple malicious actors behind Hafnium." 

Dave Bittner: So as often happens in such cases, the attack has changed from a break-in by a single organization to a riot, with many opportunistic groups smashing metaphorical windows and looting virtual organizations. The Washington Post reports that the change occurred last week. Paraphrasing security firm Volexity's Steven Adair, the Post says other unidentified actors, some of them no doubt simple criminal gangs, have raced to join the exploitation. 

Dave Bittner: As Adair put it to the Post, quote, "they went to town and started doing mass exploitation - indiscriminate attacks compromising exchange servers, literally around the world, with no regard to purpose or size or industry. They were hitting any and every server they could." 

Dave Bittner: That kind of indiscriminate approach isn't entirely consistent with espionage, although a lot of collection can be indiscriminate since things like credentials and PII could always come in handy, and you never know. But it is consistent with multiple actors, many of them criminal, taking advantage of the same vulnerabilities. It's also consistent with an intelligence service acting to get as much as it can while the getting's good before patching shuts them out. 

Dave Bittner: Many of the victims in the US have included small and medium-sized businesses, local governments and schools. As the National Security Council tweeted over the weekend, simple patching isn't enough. Affected organizations must find and eject any of the web shells the attackers left behind. 

Dave Bittner: The number of victims is very large, and mopping up will represent a protracted challenge. WIRED quotes an unnamed security researcher as calling the number of victims astronomical. That's an exaggeration, to be sure, since the total number of potential targets is far less than the number of stars in the heavens. But it's a forgivable overstatement because that number is surely really, really large. Anyone who operates an exposed Exchange Server should assume they've been compromised and act accordingly. 

Dave Bittner: Reuters reports that the White House has warned that the incident is a serious one, with an official saying, quote, "this is an active threat still developing, and we urge network operators to take it very seriously." The U.S. administration is forming a task force to organize a whole-of-government response to the cyber operations, CNN says. According to The New York Times, Deputy National Security Adviser for Cyber and Emerging Technology Neuberger is said to be leading that effort.  

Dave Bittner: Chinese operators have been busy elsewhere, too. ZDNet reports that it's not just the Russians who got busy with SolarWinds. Secureworks' Counter Threat Unit has detected what appears to be a Chinese threat actor - Secureworks calls it Spiral - using compromised SolarWinds servers to deploy the web shell Sunburst. 

Dave Bittner: The furor over the Hafnium operation comes on top of the earlier and continuing furor over the SolarWinds compromise and related cyber-espionage efforts. These have been generally attributed to Russian operators, and the U.S. is said to have begun preparing a response that the press is calling retaliation. The Chinese operation may be bigger, at least in terms of the number of organizations affected, but both are regarded as very serious. 

Dave Bittner: The New York Times quotes US National Security Adviser Sullivan on the range of potential U.S. responses. Sullivan observes that some of the response may not be particularly visible to the larger world. He said, quote, "I actually believe that a set of measures that are understood by the Russians but may not be visible to the broader world are actually likely to be the most effective measures in terms of clarifying what the United States believes are in bounds and out of bounds and what we are prepared to do in response," end quote. 

Dave Bittner: In any case, the response to both Russia and China will probably involve the imposition of a familiar range of costs. Economic sanctions will almost certainly be used, although in Russia's case, it's unclear just how much remains to be sanctioned, and other measures will, in all likelihood, include indictments, naming and shaming. They are also likely to involve some sort of retaliatory cyber operation. 

Dave Bittner: Both the Russian and Chinese operations are unusually troubling because they represent at least a potential threat that extends beyond intelligence collection. That's serious enough, but the possibility of data corruption or destruction are more serious, and the potential to compromise systems in ways that might make attacks on control systems possible is more serious still. There's no direct, publicly available evidence that these more destructive operations have occurred, but they represent a risk that affected governments cannot ignore. 

Dave Bittner: Among the responses to the SolarWinds compromise is likely to be an essentially defensive executive order aimed at preventing similar attacks in the future. The White House did signal on Friday that an executive order was under preparation to induce software developers to build greater security into their products. 

Dave Bittner: CyberScoop reports that Deputy National Security Adviser Neuberger told a SANS summit that the proposed executive order, quote, "will focus on building in standards for software, particularly software that's used in critical areas. The level of trust we have in our systems has to be directly proportional to the visibility we have. And the level of visibility has to match the consequences of the failure of those systems," end quote. 

Dave Bittner: And, finally, not to further harsh your buzz, income tax time is approaching in several parts of the world. But there's some news to make that annual news, if not good - I mean, let's be realistic - at least less bad. The National Cyber Security Alliance and the Internal Revenue Service have published some advice on how both businesses and individuals can stay safe during a period when scammers are traditionally active. You can find it at staysafeonline.org. And happy filing. 

Dave Bittner: And joining me once again is Rick Howard. He is the CyberWire's chief analyst and also our chief security officer. Rick, great to have you back. 

Rick Howard: Thanks, Dave. 

Dave Bittner: So over the last few episodes of your "CSO Perspectives" show, you've been looking at how to secure the various cloud provider networks that are out there. But, OK, you're an old Palo Alto Networks guy. 

Rick Howard: Oh, no. Ratted out. You found me out. 

(LAUGHTER) 

Dave Bittner: I don't think that's a big secret in the industry. But I was wondering, are you going to get around to addressing some of the third-party solutions - I mean, you know, security solutions that don't actually come from the cloud providers? 

Rick Howard: Well, that's a good point 'cause, you know, you might be surprised to learn this, Dave, but most of us don't store and process our data in a single-cloud environment. You know, who knew? OK? I mean, just, you know, using the CyberWire as an example, we use AWS, but we also use, like, 25 other SaaS applications, and not to mention our backup systems back at headquarters. 

Dave Bittner: Right. 

Rick Howard: And we are relatively small compared to, you know, big government and big academic and big commercial. Those organizations have data scattered all over the world. 

Dave Bittner: So help me understand here. Is what you're saying that using a cloud provider's security tool set - is that just adding another layer of complexity to I think what we can all agree is a pretty tangled security ecosystem? 

Rick Howard: That's right. You know, and then in a time when we should all be looking to reduce complexity, because the more complex it is, the more difficult it is to maintain, adding another layer of security tools from the cloud provider that doesn't easily integrate with the rest of your security ecosystem may not be the best solution for you. In fact, before we all started moving to the cloud some 10 years ago, orchestrating this ecosystem was really hard to do. Now that most of us are working in at least one cloud environment, orchestration is even harder. 

Rick Howard: So on this show, we take a look at how some of the big security platforms like Fortinet, Cisco, Check Point and Palo Alto Networks - you know, my alma mater - might be the best security solution not just for your cloud environments, but for wherever you store and process your data because of their innate ability to orchestrate across all of those environments with a single policy. 

Dave Bittner: All right. Interesting. Well, I'm looking forward to checking that out. That is "CSO Perspectives." It is part of CyberWire Pro. You can find out all about that on our website, thecyberwire.com. Rick Howard, thanks for joining us. 

Rick Howard: Thanks, Dave. 

Dave Bittner: And joining me once again is Robert M. Lee. He is the CEO at Dragos. Rob, it's always great to have you back. I wanted to check in with you and kind of get a ground truth reality check when it comes to this recent incident with the water system in Oldsmar, Fla. Can you give us your perspective on what exactly went down here? 

Robert M. Lee: Yeah. And, you know, the interesting thing to start off with is I don't know that we actually fully know what went down. So the public reporting is that there was an internet-connected human-machine interface. This is a software that operators will use to, you know, click on buttons to open valves or kick on pumps or so forth. That's a view into the operations that allows you to control it. 

Robert M. Lee: And the story goes that an adversary remotely accessed the internet-exposed HMI - they didn't have a firewall and so forth - and that with that HMI, they tried to dump lye into the water, which would then, obviously, poison or significantly hurt people at a minimum, would probably kill some folks in the community. And so that's a huge deal in any means. I'm not going to underplay this. I mean, it was an attack on a facility, and it was attacked from a safety perspective, not on a safety system. They were trying to hurt people on American soil. That's insane. 

Robert M. Lee: Why I say I don't know we fully know the details is some of the insights that have come out in some of the FBI reporting has been a little bit conflicted on some of the technical details. And some of the things they're highlighting - like, oh, they had Windows 7 software, shared credentials - have nothing to do with the attack they described, but yet they were really concerned about it. And so I'm not so sure that they're capturing necessarily what had happened there. 

Robert M. Lee: We're investigating some stuff ourselves, and so I need to shy away from the topic just a little bit. But I'll say that I think there are multiple scenarios that took place. We need to hone in on what exactly took place so we can provide reasonable recommendations out to the community. But either way, a remote adversary did try to dump lye in the water and hurt people, and that is a big deal. 

Dave Bittner: What about the notion that - as you say, you know, this whole thing is a bit insane. But isn't cranking up the sodium hydroxide into the water - I mean, isn't that kind of insanely noisy? Like, if you want to draw attention to yourself, you don't crank the dial up to 11. 

Robert M. Lee: I don't know. It depends. And so it's kind of depending on the system and the operator and everything else. So there's some environments, as an example, where regardless of what you say in the PLC, regardless of (unintelligible) in the HMI, the valve can't even support that. So you can crank it up to 11,000, you know, parts per milliliter, but it's not going to actually do that. Maybe instead of 1:100 scale it's a 1:10 scale. But that's not universal. You know, some sites you can. 

Robert M. Lee: At some sites, if you did this, an alarm would trip and there'd be a literal sound in the plant, and operators would figure it out real quick. At some sites, it'd be a little blinking light on an HMI with another 50 blinking lights, and they may not see it. And so the reality is there's not any way to generalize everything at every plant. And there are some water facilities that this exact same style of attack would have done very little, if anything. And there's some water facilities that this exact same style of attack would have significantly hurt and possibly killed people. 

Dave Bittner: Yeah. I guess my thinking is that, you know, you would imagine that someone would be a little more - they would take more steps. They'd say, OK, I'm going to change this a little bit, see if anybody notices. I'm going to change this a little bit, see if anybody notices. 

Robert M. Lee: But why? I mean, if you've got access... 

Dave Bittner: Yeah. 

Robert M. Lee: If you got access... 

Dave Bittner: If your point is harm, yeah, why hold back? 

Robert M. Lee: Yeah. I think a lot of folks - and, look; it's a good question. But I think a lot of folks try to rationalize, well, if I was the adversary here - you're not. I mean, stop it there. Well, I think - well, are you the adversary? No? OK. All right. Well, then stop mirror-imaging the adversary. They've got their own motivations. They've got their own experience. They've got their own understanding of the problem. Sometimes they do stupid things. Sometimes they do stupid things that we think are stupid that are actually smart things. We just don't know. 

Robert M. Lee: And I saw a lot of commentary on social media about that. Well, if I did this, I would've done these three ways, and this is obviously a basic threat to do it this way. No, it's not obvious. Like, it could be a sophisticated actor that did this. It could be a criminal actor that's domestic. Who knows? But you can't look at what they did with an HMI and predict the sophistication of the actor 'cause sometimes really basic stuff is all that's required. 

Dave Bittner: No, it's a really good point. I mean, it's so easy to fall into, you know, reading the tea leaves that, oh, they went after a small town instead of a big city and, you know, all those... 

Robert M. Lee: Yeah, Tampa. Therefore, it's related to the Super Bowl. And what they were really trying to accomplish - and, oh, my gosh. Like, it's like multiple levels of analytical leaps. And it's - yeah, yeah. 

Dave Bittner: So where do we go from here? I mean, how does this inform, you know, how we consider these sorts of vulnerabilities going forward? 

Robert M. Lee: I don't think there's any serious professional in this industry, especially working in IT and security, that is shocked at the state of many of our infrastructure sites. There are so many of our infrastructure operators - asset owners and operators that are doing such amazing work. When you talk about 55,000 municipal water systems where they might not even have an IT person, let alone a security person, and they're under-resourced and understaffed and everything else, like, it's not a shock. And so I don't know that you massively fix this in any one way. 

Robert M. Lee: We need to have serious conversations about how we invest in our infrastructure, how we think about technology, how we think about workforce development, how we think about engineering, training to design out some of the security risk. There's a lot of stuff to think about on a macro level, but the reality is there's nothing new that's kind of informative about the style. 

Robert M. Lee: It might be new in helping people understand that these things are going to happen, that, yeah, it's going to happen on American soil. There's a lot of, like, well, that didn't happen - like the Ukraine attack. I remember when Ukraine had happened. You know, the community eventually got around it. For the first, like, six months, it was, yeah, that was Ukraine. That's not us. Like, it could happen here. Well, yeah, that was Ukraine. And then traces (ph) happened. That's Saudi Arabia. And so sometimes we fall into that. And it's - and so the fact that this happened in the U.S. is in some ways a wake-up call to some. 

Robert M. Lee: But again, my point being these things are going to happen more. The more our infrastructure gets connected, the more adversaries get focused on it, the more frequently we are going to see IT attacks (ph). I've talked about this kind of trend for years and what we're anticipating. And we need to think about organizational and institutional change with strategies behind it, not point security solutions, not, well, if they were just using multifactor, it'd have been solved, or if they were just doing this one thing, it would've been solved. No, it wouldn't have been. 

Robert M. Lee: I don't know the folks at Oldsmar, but I know plenty of folks in the water industry. And for some of them, there's EPA regulations of, hey, if a pump fails, you got to be able to access it within a 30-minute window, but they live an hour and a half away, but they live an hour and a half away because they've got 15 plants to monitor, and they're the guy on call, and so the only way to get to it is remote-access software. Or, you know, there's this - I hate the, these guys are morons, and screw the water people. Like, nine times out of 10, they're just doing the best they can to keep the water on and to keep it clean, to keep it going. So I don't like the victim-blaming crap. 

Robert M. Lee: But I think we're silly if we think there's a simple answer. And we need to sit down and have that conversation and what that looks like, put out a strategy and go approach it 'cause it's going to get worse. It's not going to be "Die Hard." Stop freaking out about all the scenarios. But it's going to lead to death. It's going to lead to environmental impact. 

Robert M. Lee: And Oldsmar was a facility with 15,000 people that depended on it. That is not a national critical infrastructure site under any consideration. That is not a significant impact on a critical infrastructure site. That is not a national security topic. But to those 15,000 people, it sure as hell is. Those are 15,000 humans. And so we've got to do better for sure. 

Dave Bittner: Yeah. All right, well, Robert M. Lee, thanks for joining us. 

Dave Bittner: Thanks to all of our sponsors for making the CyberWire possible. 

Dave Bittner: And that's the CyberWire. For links to all of today's stories, check out our daily briefing at thecyberwire.com. And for professionals and cybersecurity leaders who want to stay abreast of this rapidly evolving field, sign up for CyberWire Pro. It'll save you time and keep you informed. Built better to ride better. Listen for us on your Alexa smart speaker, too. 

Dave Bittner: The CyberWire podcast is proudly produced in Maryland out of the startup studios of DataTribe, where they're co-building the next generation of cybersecurity teams and technologies. Our amazing CyberWire team is Elliott Peltzman, Puru Prakash, Kelsea Bond, Tim Nodar, Joe Carrigan, Carole Theriault, Ben Yelin, Nick Veliky, Gina Johnson, Bennett Moe, Chris Russell, John Petrik, Jennifer Eiben, Rick Howard, Peter Kilpe. And I'm Dave Bittner. Thanks for listening. We'll see you back here tomorrow.