SOAR and cybersecurity first principles.

Rick Howard: Hey, everyone. Rick here.

Rick Howard: SOAR stands for security orchestration, automation and response. Gartner coined the term back in 2017, but security leaders and pundits, like Jon Oltsik at CSO Magazine, started talking about the concept as far back as 2015. 

Rick Howard: The problem that we were trying to solve was how to automate the handling of all the messages, alerts and intelligence products we were receiving from the technology within our security stack. We were overwhelmed with the volume of these things that had exploded exponentially in recent years. We were manually processing them in the security operations center, or SOC, with our traditional Tier 1 and Tier 2 analysts, and we couldn't keep up. To understand why, we need to do a little cybersecurity history. 

Rick Howard: My name is Rick Howard. You are listening to "CSO Perspectives," my podcast about the ideas, strategies and technologies that senior security executives wrestle with on a daily basis. On this show, we are talking about SOAR as an idea and how it fits into the DevOps and DevSecOps movements and asking the question, why aren't you moving faster down these lanes? 

Rick Howard: Back in the early internet days for me, circa the early 1990s, the prevailing cybersecurity strategy that most network defenders pursued was something called defense in depth. The idea was that you would wrap your digital assets inside overlapping, concentric circles of defensive technology. If the bad guys got by one of the defensive circles, they would immediately run into the next. In theory, this could go on until either the network defender ran out of circles or the bad guy got tired of trying to penetrate them. 

Rick Howard: That all sounded good on paper, but what most people installed were just three circles of security technology - a firewall, an intrusion detection system and an antivirus system. What we discovered was that those overlapping circles weren't that overlapping. There were huge gaps between them that a motivated cyber-adversary could easily sneak past. 

Rick Howard: What we understood later was that defense in depth was a decent general-purpose strategy designed to prevent common bad guy behavior as a best practice. We just couldn't use it to defeat the precise tactics, techniques and procedures of any specifically known adversary. 

Rick Howard: Now, I'll probably get some pushback for this, but defense in depth, with its three-tool security stack, eventually morphed into zero trust with a completely different tool set. In other words, the tools were different, but the concept was the same - reduce the attack surface as much as possible by limiting what employees, customers and contractors can access. 

Rick Howard: Fast-forward to 2010 and the publication of the now-famous Lockheed Martin paper on intrusion kill chain prevention. This was revolutionary. Instead of a general-purpose defensive strategy like defense in depth, or perhaps in conjunction with it, we would deploy prevention obstacles at every stage of the attack sequence, designed specifically for all known bad guy attack campaigns. We wouldn't rely on hope that the bad guys would run into one of our general-purpose security circles. Instead, we would deploy obstacles on purpose where we knew the bad guys had to go, designed explicitly for their known adversary behavior. Brilliant. 

(SOUNDBITE OF ARCHIVED RECORDING) 

Unidentified Person: Winner, winner, chicken dinner. 

Rick Howard: We all thought that this was going to give the network defender the edge, that it was just a matter of time before we got this cybersecurity situation under control. We couldn't have been more wrong. 

Rick Howard: While intrusion kill chain prevention is still a key cornerstone strategy for any modern infosec program, along with zero trust, resilience and risk forecasting, the unintended consequence of the intrusion kill chain strategy was that the security vendor community came out of the woodwork to provide the tools to meet the need. And as a result, we are all managing more tools. Instead of network defenders providing the care and feeding for only three concentric circles of protection, some had a lot more. 

Rick Howard: And it's my experience that many small organizations are now managing at least 15 tools in their security stack. Medium-sized groups have upwards of 50. Fortune 500 businesses and government organizations have at least a hundred, and many have well over 200. 

Rick Howard: At the 2020 RSA Conference earlier this year, there were over 650 security vendor companies exhibiting their wares on the main showroom floor. And that doesn't account for the security vendors who weren't there. That's a lot of tools. And every one of those tools that have been deployed in our security stack is constantly spewing alerts and messages into the SOC. 

Rick Howard: At my last CSO gig, our internal SOC was trying to process over a billion alerts every quarter using the same manpower that we had when we only had three tools in the security stack. As Channing Tatum said in one of my favorite movies, "Kingsman: The Golden Circle"... 

(SOUNDBITE OF FILM, "KINGSMAN: THE GOLDEN CIRCLE") 

Channing Tatum: (As Tequila) That dog don't hunt. 

Rick Howard: By 2015, network defenders started to complain to their trusted vendors that they were tired of - air quotes here - "orchestrating" all of their alerts without any help from them. They started to kvetch that the vendors should start orchestrating that telemetry information for them so that they could concentrate on more strategic matters. One of the technologies that emerged from that chaos discussion is security orchestration, automation and response, or SOAR. 

Rick Howard: Before I describe what SOAR technology does, let's first discuss the typical security data flow for most organizations of any significant size within their SOC. 

Rick Howard: As I said at the top of the show, most infosec programs have some notion of a security stack, with one to n security tools deployed in the environment for maximum benefit. Each of those tools sits on the network, collecting telemetry that will help the tool fulfill its purpose. SOC analysts can log in to each tool, in turn, to understand what the tool is seeing and make security alerting and blocking decisions with that intelligence. 

Rick Howard: As the n gets larger, though, that gets more cumbersome to do. Logging in to a hundred tools and then trying to make sense of all of the different data elements is not for the faint of heart. 

Rick Howard: To help, infosec programs started deploying SIEMs, or security information and event management systems, another tool in the security stack designed to specifically collect the telemetry for all those security vendors into one place. This is essentially log management and provides a rudimentary orchestration capability that aids SOC analysts in data aggregation and correlation, alerting, retention and forensic analysis. 

Rick Howard: SOC leaders usually divide their analysts into three tiers. The first tier consists of the newbies and the interns. They don't have a lot of experience yet, so their task is to grind through all of those alerts and make decisions on what to do - ignore them, save them, process them or pass them to the second tier when they need help deciding what to do with them. 

Rick Howard: The second-tier analysts perform a similar process with the alerts they get from the Tier 1 end. These folks have more experience than the newbies and interns in Tier 1, but they don't know everything yet. If they can't figure out what to do, they pass it to the Unix graybeards in Tier 3. 

Rick Howard: These are the folks that have been around for a long time and have been there and done that. They know where all the bodies are buried. If they can't figure out what to do with it, it can't be figured out. 

Rick Howard: But even with a SIEM, you still have billions of messages coming into the SOC every quarter. They are centralized now into one system, but you still have to process them. This is where SOAR comes in. 

Rick Howard: SOAR technologies provide SOC analysts the tools to automate their decisions - ignore, delete, process or pass up the chain. Instead of a Tier 1 analyst sitting in the SOC all day long and swiping left most of the time to get rid of the same alert over and over again, spewed from the same device in the security stack, SOAR technologies can help automate that process. Instead of using a human to swipe left, you use code to do it. And, yes, I'm loosely including newbies and interns into the human category. 

Rick Howard: When we decided to use SOAR technologies in my last CSO gig, within a year, we had reduced the number of messages in the SIEM that needed to be processed by a human every quarter from over a billion - that is billion with a giant B - to just under 500. Even Unix graybeards, as slow as they move, can process 500 alerts in 90 days. 

Rick Howard: I kid. I kid. I used to be a Unix graybeard, so I kid. 

Rick Howard: If the SOC did nothing other than automatically process these messages, the technology investment would be worth it. By using a SOAR technology, you can essentially eliminate the need for Tier 1 and Tier 2 analysts. And I'm not talking about firing those newbies and interns either. I'm talking about repurposing them to more strategic tasks, like maybe automating the thousand other projects that need to be automated within the SOC. If we start to do that, then we are getting closer to DevSecOps. 

Rick Howard: The DevOps movement started around 2010 but started to gain traction when Gene Kim published his Cybersecurity Canon Hall of Fame book, "The Phoenix Project," in 2013. Sometime after, the network defender community realized that they were being left behind in this infrastructure-as-code movement. They realized that if they didn't get on the bandwagon soon with all of the other IT and networking people, they would end up being in the same position that they are in right now, mostly considered as an afterthought in any kind of software development process. 

Rick Howard: If the network defenders could somehow join forces with the DevOps community, they could leapfrog years of bureaucratic neglect and friction from the IT community by shifting left in the software development lifecycle process. And by the way, shifting left is a good thing in software development, but mindlessly swiping left in the SOC is a bad thing. 

(SOUNDBITE OF ARCHIVED RECORDING) 

Unidentified Actor #1: (As character) That's a lollipop. 

Unidentified Actor #2: (As character) Oh, that is not a lollipop. 

Unidentified Actors #1: (As characters) Left swipe. 

Unidentified Actors #2: (As characters) Left swipe. 

Rick Howard: But that's as far as it went. It was a good idea, but few have pursued it. 

Rick Howard: As in all cases, there are exceptions. But for the most part, the DevOps communities and the security communities are still sitting on opposite sides of a great divide. It's as if the security community is waiting to be invited to the DevOps community and the DevOps community is waiting for the network defenders to start writing some code. 

Rick Howard: My advice to the security community is to stop waiting for an engraved invitation. Just start. And I have the perfect place to begin. 

Rick Howard: The good news with most popular SOAR products is that they know how to connect to many of those 650 security products out there. That means your infosec team doesn't have to figure it out on their own. The SOAR vendor does the work for you. 

Rick Howard: The first step is to get busy reducing those billion alerts every quarter that your Tier 1 analysts are manually processing today. Once you get that done, the sky's the limit. 

Rick Howard: One immediate thing you could do is to use your SOAR technology to update your security stack with the proper prevention controls for a current adversary campaign. How about Fancy Bear? 

Rick Howard: In the last episode - Season 3, Episodes 3 and 4 - I mentioned how big a fan I am of the MITRE ATT&CK framework. It houses the most comprehensive open-source collection of adversary tactics, techniques and procedures in the world right now. So go to the MITRE ATT&CK webpage. Look up all the mitigations for Fancy Bear. And there will be a lot. For every tool in your security stack, decide how to implement the recommended Fancy Bear mitigations for it. Now, you will not be able to implement every mitigation for every tool at all phases of the intrusion kill chain. That's OK. Do as many as you can. 

Rick Howard: Then, fire up your SOAR tool. Automate the delivery of the Fancy Bear mitigation set to your deployed security stack. If you get that done, move to the next adversary group. How about Electric Panda? 

Rick Howard: This model has one big advantage. Instead of individually implementing the Fancy Bear and Electric Panda mitigation set for every tool in the security stack and probably losing track of it all because you have a hundred tools in your stack, you centrally manage it with your SOAR tool. When changes to the mitigation set occur later, you have one place to go to make the updates. When Fancy Bear and Electric Panda update their attack sequences, you have only one place to look to determine if your organization is vulnerable. If you get this done, you have essentially automated your defensive playbook for two known adversary campaigns. The next step is to rinse and repeat. 

Rick Howard: According to the Cyber Threat Alliance, a cyberthreat intelligence sharing group for security vendors, there are roughly just over a hundred active adversary groups working on the internet today. The MITRE ATT&CK framework tracks the bulk of them in terms of potential mitigations. A hundred active adversary groups is not that many. In a small amount of time, one coder in the SOC could build the network defensive campaign framework for all known adversary groups. This is a worthy DevSecOps project, and one that we all need to pursue. 

Rick Howard: SOAR is a nice label on which to hang all things involved with automating tasks within the SOC. You can use a SOAR vendor, or you can just roll your own. The point is to start building your own code as infrastructure now. By taking this first step, you can get out from under all of the noise that those billion alerts every quarter were causing, free up some personnel assets at the entry level and redirect them to more strategic pursuits. All of that is good and positive. As you go, your team is learning about DevOps as a philosophy. Once you get comfortable with it, start reaching out to your DevOps team on the other side of the aisle and insert yourself into a shift-left mindset for the development process. 

Rick Howard: And that's a wrap. If you agree or disagree with anything I have said, hit me up on LinkedIn or Twitter, and we can continue the conversation there. Next week, I have invited our pool of CyberWire's experts to sit around the Hash Table to discuss how they think about SOAR and orchestration and DevSecOps. You won't want to miss that. 

Rick Howard: The CyberWire's "CSO Perspectives" is edited by John Petrik and executive produced by Peter Kilpe. Our theme song is by Blue Dot Sessions, remixed by the insanely talented Elliott Peltzman, who also does the show's mixing, sound design and original score. And I am Rick Howard. Thanks for listening.