CSO Perspectives is a weekly column and podcast where Rick Howard discusses the ideas, strategies and technologies that senior cybersecurity executives wrestle with on a daily basis.
SOAR – a first principle idea.
SOAR stands for “security orchestration, automation and response.” Gartner coined the term back in 2017, but security leaders and pundits, like John Oltsik at CSO Magazine, started talking about the concept as far back as 2015. The problem that we were trying to solve was how to automate the handling of all of the messages, alerts, and intelligence products we were receiving from the technology within our security stack. We were overwhelmed with the volume of these things that had exploded exponentially in recent years. We were manually processing them in the security operations center, or SOC, with our traditional tier 1 and tier 2 analysts, and we couldn’t keep up.
To understand why, we need to do a little cybersecurity history.
A digression into some cybersecurity history.
The prevailing strategy that most network defenders used back when the internet was young in the early 1990s was something called “defense in depth.” The idea was that you would wrap your digital assets inside overlapping concentric circles of defensive technology. If the bad guys got by one of the defensive circles, they would immediately run into the next. In theory, this could go on until either the network defender ran out of circles or the bad guy got tired of trying to penetrate them.
That all sounded good on paper, but what most people installed were just three circles of security technology: a firewall, an intrusion detection system, and an antivirus system. What we discovered was that those overlapping circles weren’t that overlapping. There were huge gaps between them that a motivated cyber adversary could easily sneak past. What we understood later was that defense in depth was a decent general purpose strategy designed to prevent common bad guy behavior as a best practice. We just couldn’t use it to defeat the precise tactics, techniques, and procedures of any specific known adversary. I will probably get some pushback for this, but defense in depth, with its three-tool security stack, eventually morphed into zero trust with a completely different tool set. In other words, the tools were different but the concept was the same: reduce the attack surface as much as possible by limiting what employees, customers, and contractors can access.
Fast forward to 2010 and the publication of the now famous Lockheed Martin paper on intrusion kill chain prevention. This was revolutionary. Instead of a general purpose defensive strategy, or perhaps in conjunction with one, we would deploy prevention obstacles at every stage of the attack sequence, designed specifically for all known bad guy attack campaigns. We wouldn't rely on hope that the bad guys would run into one of our general purpose security circles. We would deploy obstacles on purpose where we knew the bad guys had to go, designed explicitly for their known adversary behavior. Brilliant! We all thought that this was going to give the network defender the edge, that it was just a matter of time before we got this cybersecurity situation under control. We couldn’t have been more wrong.
While intrusion kill chain prevention is still a key cornerstone strategy for any modern infosec program―along with zero trust, resilience, and risk forecasting―the unintended consequence of the strategy was that the security vendor community came out of the woodwork to provide tools to meet the need. As a result, we are all managing more tools. Instead of network defenders having to feed and care for three concentric circles of protection, some had a lot more. It’s my experience that many small organizations are now managing at least 15 tools in their security stack. Medium-sized groups have upwards of 50. Fortune 500 businesses and government organizations have at least 100 and many have over 200. At the 2020 RSA Conference, there were over 650 security companies exhibiting their wares on the main showroom floor. And that doesn’t account for the security vendors who weren’t there.
That’s a lot of tools.
And everyone of those deployed tools is constantly spewing alerts and messages into the SOC. At my last CSO gig, our internal SOC was trying to process over a billion alerts every quarter using the same manpower that we had when we only had three tools in the security stack. As they say in my home state of South Dakota, that dog just ain't gonna hunt.
By 2015, network defenders started to complain to the vendors in their security stack that they were tired of “orchestrating” all of their alerts without any help. They started to kvetch that the vendors should start orchestrating that telemetry information for them so that they could concentrate on more strategic matters. One of the technologies that emerged from that chaos discussion is security orchestration, automation and response.
What does SOAR do?
Before I describe what SOAR technology does, you must first understand the typical security data flow for most organizations of any significant size. As I described above, most infosec programs have some notion of a security stack with 1 to n security tools deployed in the environment for maximum benefit. Each of those tools sits on the network, collecting telemetry that will help the tool fulfil its purpose. SOC analysts can log in to each tool, in turn, to see what it is seeing and make security decisions with that intelligence. As the n gets larger though, that gets more cumbersome to do. Logging in to a hundred tools and then trying to make sense of all of the different data elements is not for the faint of heart.
To help, infosec programs started deploying SIEMs, or security information and event management systems, another tool in the security stack, designed to specifically collect the telemetry from all of those security vendors into one place. This is essentially log management and a rudimentary orchestration capability that aids SOC analysts in data aggregation and correlation, alerting, retention, and forensic analysis.
SOC leaders usually divide their analysts into three tiers. The first tier consists of the newbies and the interns. They don’t have a lot of experience yet so their task is to grind through all of those alerts and make decisions on what to do: ignore them, save them, process them, or pass them to the second tier. The second tier analysts perform the same process with the alerts they get from the tier 1 analysts. If they can’t figure out what to do, they pass it to the Unix greybeards in tier 3. These are the folks that have been around for a long time and have been there and done that. They know where all of the bodies are buried. If they can’t figure out what to do with it, it can’t be figured out.
But even with a SIEM, you still have billions of messages coming into the SOC every quarter. They are centralized now in one system, but you still have to process them. This is where SOAR comes in. SOAR technologies provide SOC analysts the tools to automate their decisions: ignore, delete, process, or pass up the chain. Instead of a tier 1 analyst sitting in the SOC all day long and swiping left most of the time to get rid of the same alert over and over again, spewed from the same device in the security stack, SOAR technologies can help automate that process. Instead of using a human to swipe left, you use code to do it. And yes, I am loosely including newbies and interns in the human category.
When we decided to use SOAR technologies in my last CSO gig, within a year, we had reduced the number of alerts that needed to be processed by a human every quarter from a billion to 500. Even Unix greybeards, as slow as they move, can process 500 alerts in 90 days.
The relation between SOAR and DevSecOps.
If the SOC did nothing other than what I described above, the technology investment would be worth it. By using a SOAR technology, you can essentially eliminate the need for tier 1 and tier 2 analysts. And I’m not talking about firing those newbies and interns. I am talking about repurposing them to more strategic tasks like maybe automating the thousand other projects that need to be automated within the SOC. If we start to do that, then we are getting closer to DevSecOps.
The DevOps movement started around 2010 but started to gain traction when Gene Kim published his Cybersecurity Canon Hall of Fame book, “The Phoenix Project” in 2013. Sometime after, the network defender community realized that they were being left behind in this infrastructure-as-code movement. They realized that if they didn’t get on the bandwagon with all the other IT and networking people, they would end up being in the same position that they are in now, mostly considered as an afterthought in any kind of software development process. If the network defenders could somehow join forces with the DevOps community, they could leapfrog years of bureaucratic neglect and friction from the IT community by shifting left in the software development lifecycle process. And by the way, shifting left is a good thing in software development, but mindlessly swiping left in the SOC is a bad thing.
But that’s as far as it went. It was a good idea, but few have pursued it. As in all cases, there are exceptions, but for the most part, the DevOps communities and security communities are still sitting on opposite sides of a great divide. It’s as if the security community is waiting to be invited to the DevOps community, and the DevOps community is waiting for the security community to start writing some code. My advice to the security community is to stop waiting for an engraved invitation. Just start. And I have the perfect place to begin.
The good news with the most popular SOAR products is that they know how to connect to many of the 650 security products out there. That means your infosec team doesn’t have to figure it out on their own. The SOAR vendor does the work for you. The first step is to get busy reducing those billion alerts every quarter that your tier 1 analysts are manually processing today. Once you get that done, the sky's the limit. One immediate thing you could do is to use your SOAR technology to update your security stack with the proper prevention controls for a current adversary campaign. How about Fancy Bear?
SOAR and DevSecOps: defending against Fancy Bear's cyber campaigns.
Go to the MITRE ATT&CK webpage. Look up all the mitigations for Fancy Bear. There will be a lot. For every tool in your security stack, decide how to implement the recommended Fancy Bear mitigations for it. You will not be able to implement every mitigation for every tool at all phases of the intrusion kill chain. That’s OK. Do as many as you can. Then, fire up your SOAR tool. Automate the delivery of the Fancy Bear mitigation set to your deployed security stack. If you get that done, move to the next. How about Electric Panda?
This model has one big advantage. Instead of individually implementing the Fancy Bear and Electric Panda mitigation set for every tool in the security stack, and probably losing track of it all because you have a hundred tools in your stack, you centrally manage it with your SOAR tool. When changes to the mitigation set occur later, you have one place to go to make the updates. When Fancy Bear and Electric Panda update their attack sequences, you have only one place to look to determine if your organization is vulnerabile. If you get this done, you have essentially automated your defensive campaign for two known adversary campaigns. The next step is to rinse and repeat.
According to the Cyber Threat Alliance, a cyber threat intelligence sharing group for security vendors, there are roughly just over a hundred active adversary groups in existence today. The MITRE ATT&CK framework tracks the bulk of them in terms of potential mitigations. A hundred active adversary groups is not that many. In a small amount of time, one coder in the SOC could build the network defensive campaign framework for all known adversary groups with your SOAR tools. That is a worthy DevSecOps project and one that we all need to pursue.
Final Thoughts on SOAR.
SOAR is a nice label on which to hang all things involved with automating tasks within the SOC. You can use a SOAR vendor or you can just roll your own. The point is to start building your own code as infrastructure now. By taking this first step, you can get out from under all of the noise that those billion alerts every quarter were causing, free up some personnel assets at the entry level, and redirect them to more strategic pursuits. All that is good and positive. As you go, your team is learning about DevOps as a philosophy. Once you get comfortable with it, start reaching out to your DevOps team on the other side of the isle and insert yourself into a shift-left mindset for the development process.
“Cybersecurity First Principles: DevSecOps.” Rick Howard, CSO Perspectives, The CyberWire, 8 June 2020.
“FAQ,” RSA Conference, 2020.
"Intelligence-Driven Computer Network Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains,” Eric Hutchins, Michael Cloppert, Rohan Amin, Lockheed Martin Corporation, 2010, last visited 30 April 2020.
“Malware? Cyber-crime? Call the ICOPs!” Jon Oltsik, CSO, Cybersecurity Snippets, 22 June 2015.
“Market Guide for Security Orchestration, Automation and Response Solutions,” Gartner, ID G00727304, 21 September 2020.
“MITRE ATT&CK,” Mitre.
“The Cybersecurity Canon: The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win,” book review by Rick Howard, Palo Alto Networks, 21 October 2016.
“The Cyber Kill Chain is making us dumber: A Rebuttal,” Rick Howard, LinkedIn, 29 July 2017.
“The Evolution of SOAR Platforms,” Stan Engelbrecht, SecurityWeek, 27 July 2018.
“What is SOAR (Security Orchestration, Automation, and Response)?” Kevin Casey, The Enterprisers Project, 30 October 2020.