SolarWinds through a first principle lens.

Rick Howard: Hey, everyone - Rick here. Welcome to Season 4 of the "CSO Perspectives" podcast. We have some interesting things planned for you this season. Of course, we're going to touch on the SolarWinds attack campaign. It will give us a chance to test our first principle theories and strategies. But we're also going to assess those same strategies for each of the big three cloud platforms - the Google Cloud Platform, or GCP; Microsoft Azure and Amazon Web Services, or AWS. I'm basically learning as I go. I can spell GCP, AWS and Azure correctly, you know, 3 times out of 5. But that's as far as it goes. I am one inch deep here. So buckle in, and join me as we try to dig a little deeper on all of these topics.

Rick Howard: My name is Rick Howard. You are listening to "CSO Perspectives," my podcast about the ideas, strategies and technologies that senior security executives wrestle with on a daily basis. For the past year, I've been developing a set of cybersecurity strategy theories based on first principles. The key word in that description is, quote, "theories," unquote. After years of running around in the industry, working as a security executive or advising the same, I have seen many things tried. Some have worked, and some have failed. But that experience has allowed me to build this grand unifying theory about how to do cybersecurity in the future. The question that I've been asking myself this past year is, would it work? Would that set of theories prevent material impact in the real world? Would they have worked against the latest SolarWinds attack? Let's find out. 

Rick Howard: Let's just say I'm the CSO of a medium-to-large-sized organization. With a nod towards the "Iron Man" and "Avengers" movies, let's call the company Stark Enterprises - you know, because I'm a nerd geek of the highest order. For the past year, the Stark Enterprise infosec team has pursued such goals as resilience, intrusion kill chain prevention, zero trust and risk forecasting. Although many more things remain to be done tactically, the executive team - you know, Tony Stark and Pepper Pots - have been pleased with the overall progress. For the sake of this exercise, assume we've deployed at least 50% for all four strategies. And then the SolarWinds incident hits us. Would those deployed first principle strategies have prevented a material impact to Stark Enterprises? 

Rick Howard: The SolarWinds event has collected a growing list of campaign and adversary group names from various commercial security vendors - Dark Halo named by Volexity, SolarStorm named by my old friends at Unit42, Solorigate named by Microsoft and UNC2452 named by FireEye. For the purposes of this podcast, I will use 2452 to refer to the adversary group and SolarStorm to refer to the campaign. Unfortunately, there have also been unsubstantiated preliminary reports attributing the attacks to APT29 - you know, Cozy Bear, aka the Russian Foreign Intelligence Service, or SVR. The fact is that the SolarStorm attack sequence matches none of the Cozy Bear known campaigns. As far as I can tell, the only previously known procedure deployed by 2452 was the use of a piece of malware called Cobalt Strike. Now, according to the wiki Malpedia, quote, "Cobalt Strike is a paid penetration testing product that allows an attacker to deploy an agent named Beacon on the victim machine," unquote. 

Rick Howard: Over the years, Cobalt Strike has been used by adversary groups from Russia, Vietnam, China and Iran. It could be anybody at this point. But according to Recorded Future's John Wetzel, U.S. government officials will release technical evidence soon that will point to Russia. They haven't yet for fear of revealing sources and methods. Now, I understand about protecting sources and methods. But until somebody steps up and makes the evidence public, we just don't know. 

Rick Howard: By the way, there's a way to do this without compromising your sources and methods, and the U.S. government has done it many times in the past. They passed the technical evidence to the commercial security vendors, specifically the Cyber Threat Alliance. This group of roughly 30 security vendors with some of the best intelligence teams in the world will confirm the intelligence in their own product lines and make attribution assessments while they're doing it. This leaves the government out of it and does not reveal their sources and methods. In other words, they point the vendors in the right direction and let them draw their own conclusions. The vendors find the evidence in their own data, and the government doesn't have to reveal how their intelligence groups discovered it. The U.S. government hasn't done that yet, and that makes me suspect the validity of the Russian attribution claim. 

Rick Howard: Don't get me wrong. It's probably the Russians. There are only a handful of nation-states that have this kind of capability, and it feels like something the Russians have done in the past - you know, like Sandworm. But also, so has China. Think OPM as a supply chain attack. But today, we are just not there yet in terms of actual evidence to prove a Russian attribution, at least in the public. 

Rick Howard: The bottom line is that 2452 built the campaign almost entirely from scratch, a zero-day campaign, if you will. This is highly unusual. For the past 10 years, most adversary groups, even the stealthy ones, haven't created brand-new attack campaigns with original components for each phase of the intrusion kill chain. They haven't had to. Usually, they change the section of the attack sequence that doesn't work anymore because the victims have figured out how to block it, or the adversary group has innovated some better way to do it. To invent entirely new attack campaigns is expensive and time-consuming. In that way, SolarStorm is on the same level of complexity and resource commitment as another famous zero-day campaign, the 2010 Stuxnet campaign. That cyber operation penetrated the Iranian nuclear plant in Natanz for the purpose of degrading the production of nuclear bombs. 

Rick Howard: By creating a zero-day campaign, 2452 nullified a cornerstone pillar in our first principle cybersecurity strategy wall - intrusion kill chain prevention. That strategy leverages the observation that cyber adversaries don't typically do this, and it shifts the infosec teams away from blocking technical artifacts in isolation, like malware or zero-day exploits, with no context about what the adversary is trying to accomplish. The idea is that you deploy prevention and detection controls for every tactic, technique and procedure across the intrusion kill chain for all known campaigns. And that way, even if the adversary changes a specific TTP of the attack sequence and makes it invisible to thread hunters, the other prevention and detection controls still in place will prevent the success of the operation. 

Rick Howard: You all probably know Ryan Olson. He's a friend and colleague and currently the Palo Alto Networks Unit42 intelligence director. He and I wrote an entire paper on this concept that we published in the Cyber Defense Review this past fall. With a zero-day campaign like SolarStorm, that strategy doesn't work. Stark Enterprises doesn't have any detection or prevention controls in place to stop the success of the operation because every step in the attack sequence is unknown except for the Cobalt Strike malware. That knowledge might be enough to stop this campaign, but probably not. We have to admit it. At the time of the SolarStorm attack, our intrusion kill chain prevention strategy most likely didn't prevent material impact to Stark Enterprises. This leaves us with the other three first principle strategies - zero Trust, resilience and risk assessment. Would some combination of those have prevented a material breach? Before we can answer that, we have to understand how the SolarStorm attack sequence worked - at least the parts we know about. 

Rick Howard: In terms of the intrusion kill chain model, we know that at least one initial entry vector was a backdoor Trojan inserted into the SolarWinds Orion software platform and automatically delivered to roughly 18,000 SolarWinds customers as a software update. According to the SolarWinds website, Orion is an orchestration platform designed to manage the entire IT stack from network performance to configuration to IP management and so on. It's similar to security platforms like those offered by Fortinet, Check Point and Palo Alto Networks. But in this case, SolarWinds' Orion, it's for IT orchestration. 2452 used a supply chain attack vector as the initial delivery mechanism. 

Rick Howard: By supply chain, I mean that it didn't attack the victims directly. Instead, 2452 attacked a key software component - in this case, Orion - that potential victims were likely to use. 2452 inserted some backdoor code into Orion, rode into the victim's network via the legitimate software update channel, and then used the backdoor to compromise its intended victims. 

Rick Howard: There have been several supply chain attacks over the years. But arguably, the two best-known are, first, the 2013 Target cybercrime breach, where the supply chain partner was Fazio Mechanical Services, Target's heating, ventilation and air conditioning, or HVAC, supplier. The second one was the 2017 Ukraine supply chain attack, better known as NotPetya, where the supply chain partner was Intellect Service, the maker of a financial software package called MeDoc used by many Eastern European companies. The purpose of the supply chain attack was the Russians conducting continuous low-level cyber conflict and influence operations against Ukraine. 

Rick Howard: The other thing we know is that 2452 leveraged their victims' identity management systems. They managed to steal the secret key from an on-prem single sign-on management server responsible for federated accounts. In this case, where the victim was a Microsoft customer, 2452 leveraged the Active Directory Federation Services server. 

Rick Howard: Now, if you remember from Season 2, Episode 7 and Season 2, Episode 8, we talked about identity management and the federated model. We learned that you get this kind of transitive property of trust with federation. If the University of Michigan trusts Ohio State University and Ohio State University trusts its CISO, Helen Patton, one of our hash table subject matter experts, then the University of Michigan trusts Helen. In other words, they believe it's her when she logs in. But they still authorize her access to U of M's resources locally. She doesn't get carte blanche access just because she's Helen. U of M decides that. 

Rick Howard: 2452, through some process of Microsoft Active Directory privilege escalation, stole the private key use to sign SAML tokens. SAML stands for Security Assertion Markup Language. Again, from the same two "CSO Perspectives" episodes, SAML refers to a heavyweight XML variant language that facilitates one computer to perform both authentication and authorization on behalf of other computers. 2452 then used that secret key to forge trusted authentication tokens for cloud resources. In other words, they stole the secret key from an on-prem server to create authorization tokens for virtual systems in the cloud. Now, this SAML technique isn't new. The NSA published an advisory on it back in 2017, noting that it has been used by both nation-state groups and other types of threat actors. 

Rick Howard: Zero trust is the obvious strategy to leverage here. 2452 went after two systems that control key components of the Stark Enterprises network undercarriage, network management with the Orion orchestration platform and the identity management system with Active Directory Federation Services. Since Stark Enterprises is at least 50% complete on deploying the four pillar first principle strategies, it is likely that the risk assessment strategy identified these two systems as warranting prioritization and special attention. 

Rick Howard: With the zero trust strategy, we are trying to reduce the Stark Enterprises attack surface by limiting employee access to only the resources they need to do their job and nothing more. It seems reasonable, then, that the Stark Enterprises infosec team would have greatly limited the number of employees who have access to those two systems. It might even have restricted the accounts further to people authorized to commit changes. Additionally, they would have insisted that the accounts used by those employees weren't the accounts they use when they were doing their daily routines, like reading email and surfing the web. 

Rick Howard: And lastly, the Stark Enterprises infosec team would have probably established special monitoring for two processes - change to network configuration and creation of authorization tokens. The monitoring process would ensure that somebody else knew about those operations and could verify that the task was done with the correct procedures. Think of it like the two-person control protocols used by the Air Force and others to control the launching of nuclear missiles. The Stark Enterprises infosec team would deploy all of these zero-trust tweaks in an effort to reduce the probability of privilege escalation. That would have likely stopped the SolarStorm campaign. 

Rick Howard: On the other hand, I realize that this zero-trust plan has a lot of bureaucratic pieces to it. When the boss is breathing down your neck to get something done, most of us don't have time for that kind of nonsense. Before SolarStorm, as the CEO of Stark Enterprises, I might have assessed the probability of a zero-day campaign impacting the business as being pretty small. Supply chain attacks had happened before for sure, but they weren't common. And with my intrusion kill chain prevention strategy deployed at 50%, I might have been feeling pretty good about my chances. 

Rick Howard: The epiphany for me, though, is remembering the atomic cybersecurity first principle, the overriding essential element that drives the four pillar strategies. As the Stark Enterprises CSO, I am trying to reduce the probability of material impact due to a cyber event. For any adversary trying to cause Stark Enterprises harm, the crown jewels, so to speak, includes the identity system. If I don't lock that down, I open Stark Enterprises up to all kinds of trouble. By reducing the attack surface of network configuration changes and authorization token creation, I might prophylactically prevent many future known and unknown campaigns from being successful. In that way, establishing these bureaucratic zero-trust procedures might be the very first thing I do. 

Rick Howard: FireEye was the first commercial company that publicly admitted being a victim of the SolarStorm campaign. In Season 2, Episode 3 and 4, we talked about incident response and the executive decision about when to go public with breach information. There are basically two choices. You can announce early to get in front of the story, but risk public disfavor when additional information comes out later that might contradict your initial assessment. Or you can announce later when the information is almost perfect, but risk public disfavor by being accused of withholding important information. 

Rick Howard: The FireEye CEO, Kevin Mandia, decided to announce early. On 8 December, he told the world, and the company's stock price dropped from $15.51 to $13.54. Not a huge plunge, but it's still significant. But because of the way Mandia handled the announcement with complete transparency about how the company discovered the attack, how 2452 conducted their attack campaign against FireEye and the sharing of the indicators of compromise that others could use to hunt for 2452 in their own environments, the stock dip was a minor setback. And just before Christmas, the stock price rebounded to $24.84. So FireEye lost about two bucks on the stock price immediately after the announcement, but gained just over $9 over the price before the announcement in just a couple of weeks. 

Rick Howard: Now, the stock market is a crazy thing, and I defy anyone to explain how it really works. You know, in my last company, I remember we were all getting ready to go on Christmas break. This was a few years ago. The stock price was set at a certain value, and then everybody went on holiday. New Year's came and went. No work had been done in the interim - no major announcements or scandals or anything. But when the opening bell rang in the start of the new year, the stock price plunged by, like, $25. Explain that. I think the most you can say about the FireEye stock price is that the way that FireEye managed the breach announcement at least contributed to this favorable response. 

Rick Howard: There are other metrics that we might use to measure material impact that aren't public right now, like customer retention down the road and hitting their quarterly numbers at the next analyst call. For now, though, the way FireEye handled the announcement has been favorably received by the investment community, and the SolarStorm campaign hasn't had a material impact on the business. By executing a well thought-out incident response plan, Kevin Mandia and his team made FireEye more resilient. Microsoft followed the same playbook and got similar results. They lost about six bucks on the stock price immediately after the announcement, but gained just over $9 over the price before the announcement. 

Rick Howard: And it's interesting to note that there are potentially 18,000 other organizations that have decided to delay their breach announcement, including SolarWinds. The SolarWinds stock price has been slowly degrading since the announcement and has shown no signs of rising. As I'm saying this, their stock price has dropped about $9. Of the 11 commercial firms we know were touched by the campaign, only FireEye and Microsoft made their public announcements early. Down the road, we will see which strategy worked for these other SolarStorm victims. 

Rick Howard: I think it will be years before we know the full extent of the attack sequence executed by 2452. We knew some of the sequence details for sure, mostly because they are so unusual. But we by no means know the entire story of how the adversary group penetrated potentially 18,000 victims, nor what 2452 exfiltrated if they were successful. Going through the Stark Enterprises exercise, though, proved to me that our four-pillar cybersecurity first principles strategies are fundamentally sound. It also proved to me that you can't focus on one and not the other three. One of them, two of them, or even three of them are unlikely to be sufficient. You have to do all four in order to protect Stark Enterprises. 

Rick Howard: And that's a wrap. If you agree or disagree with anything I've said, hit me up on LinkedIn or Twitter, and we can continue the conversation there. Next week, I've invited the CyberWire's pool of experts to the hash table to discuss what they think about the SolarWinds incident and what strategies they recommend that will defend Stark Enterprises in the future. You don't want to miss that. 

Rick Howard: The CyberWire's "CSO Perspectives" is edited by John Petrik and executive produced by Peter Kilpe. Our theme song is by Blue Dot Sessions, remixed by the insanely talented Elliott Peltzman, who also does the show's mixing, sound design and original score. And I am Rick Howard. Thanks for listening.