Transform Your SOC And Get Ahead Of The Threats
Clay Brothers: I think the most important thing is to constantly think differently or continuously improve. The SOC, when you get to that, what you think is that desired future state, you've hit your roadmap for the annual goals that you have, that is not the end. Even when you get these shiny new toys and you have the right people in process, it constantly needs to evolve, right, and that is the world we live in. It is always going to change. Threat actors are constantly evolving. Having visibility into threat intelligence to be able to understand how these threat actors are evolving and being able to morph your SOC to effectively detect those is critical. It is a constant battle and you have to continuously improve to keep up with this battle. [ Music ]
David Moulton: Welcome to Threat Vector, the Palo Alto Network's podcast, where we discuss pressing cybersecurity threats and resilience and uncover insights into the latest industry trends. I'm your host, David Moulton, Director of Thought Leadership for Unit 42. [ Music ] Today, I'm speaking with Clay Brothers, Senior Director of Palo Alto Network's Unit 42. Clay is a seasoned cybersecurity leader with a laundry list of certifications and a background in transforming SOCs to modernized, AI-enabled automation centers. He brings invaluable insights to the intersection of cutting-edge technology and security operations. Today, we're going to talk about SOC transformation and staying ahead of threats. Here's our conversation. [ Music ] Clay Brothers, welcome to Threat Vector. Thrilled to have you with us today.
Clay Brothers: Thank you. Thrilled to be here. Excited about this.
David Moulton: Clay, I saw that you're a proud Virginia Tech alum, and I came across this fun fact. Apparently, the Hokies football team's entrance song is "Enter Sandman," and it's so intense it's been recorded on a seismograph. Were you ever a part of those incredibly loud entrances? And if so what was that experience like?
Clay Brothers: I did not miss one home game when I was a student. In fact, I still enjoy as many Virginia Tech football games as I can. "Enter Sandman," that entrance is world-class. It will put -- all -- give you goosebumps all over your body. It is the best.
David Moulton: I think I've even seen a video where Metallica came out as part of one of those entrances, which is just like mind-blowingly awesome that the band was there, the team was there, all the fireworks. It seems like the stadium itself was physically moving, which is a little maybe frightening, but, you know, good architecture can handle a little bit of that. So I'm glad to hear that you were a part of that and cheering, I assume, as loud as you could, but maybe not necessarily heard by everyone.
Clay Brothers: Oh, yeah. Blacksburg is a small college town, but it gets very rowdy and very loud on football Saturdays or if we're playing Thursday night or Friday night. You can't beat it.
David Moulton: Yeah, that's awesome. Well, we're not going to go into our college towns and football, but today we're going to dive into SOC transformation, what it really means, and why it's so important right now, and get some of your insights on staying ahead of a lot of these evolving threats. We've got a lot to talk about. Let's get right into it. Are you a fan of David Epstein's book, Range, that talks about this idea of having a lot of different things that you try and you put those together before you make a final decision on what you're going to be when you grow up?
Clay Brothers: Yeah, I think it's the right way to go.
David Moulton: Yeah. I suspected maybe you were, and have you hit that point where you've decided what you want to be when you grow up? Is this the final Clay Brothers formation, or are you still looking at another transformation? What's next?
Clay Brothers: I don't know if you ever figure it out. Maybe you do, but I would say no right now. I mean, over the past five years at Unit 42, I've focused, started focusing heavily in security operations and SOC, which is really where I have a lot of passion right now. You know, when I first joined, I was put into the lead to manage a SOC transformation project with a major oil and gas company, so for eight months, all day, every day, I was helping the company mature their SOC, whether it was building a SOC charter, instant response plan to actually designing playbooks for them, helping them select a SIM tool, helping them with automation and what kind of head count they should look like and structure, and I really dove really deep into SOC on that project, and ever since then, I've grown up to love those types of projects, doing purple-teaming and tabletop exercises and attack and breach simulation, and today I lead the SOC assessment service globally here at Unit 42.
David Moulton: Can you help me out with something? What does SOC transformation, it's something I see a lot, what does that mean in today's context? And in your opinion, why is it so essential for businesses to prioritize this now?
Clay Brothers: Yeah, so SOC transformation is all about getting from where you are today to where either you want to be or need to be in the future. Generally, what we see is companies have what we call a traditional SOC model where they have a legacy SIM tool that's collecting data from disparate parts of the environment, and you have static signature-based or detection-based use cases that are triggering alerts for the SOC that they need to go manually triage, analyze, mitigate, remediate, close. From there, Unit 42 obviously sees a lot in terms of the incidence. We know that threat actors are increasing their scale and sophistication and speed, which is driving a need to think differently. Even with AI today, it's so easy to morph an attack to avoid a detection, a static detection, and so these SOCs are forced to think differently and change the way they're structured.
David Moulton: Now, as you're going through this, you're talking about speed, scale, and sophistication, and when I was looking at our global incident response report, it seems that speed scale and sophistication has moved to an even more aggressive or destructive type of behavior, and I got to think that is a big business driver when you're going, like, it's not just part of your network that's been locked up, but it's your network is locked up. Your data is locked up. There is a harassment going on with your customers. There's a harassment going on with your employees. There's an insistence on shutting down, not just your business, but part of your overall supply chain. That all, to me, becomes this motivating moment when you're dealing with a threat actor. How does transformation help with that, or is that specifically why businesses need to transform what they're doing in their security and their security operations center?
Clay Brothers: Yeah, it's a huge part of it. So traditionally, the data, the security data from -- in the endpoint is kind of stuck in an EDR tool. The cloud's stuck in whatever cloud tool you're using. Your identity is in your IAM tool, right? And it's in these siloed systems that makes it really difficult for the SOC to be able to effectively detect and respond to those attacks. Just as you mentioned, you know, in our incident response report, we're seeing that 84% of incidents are attacking multiple fronts, right? Human, identity, network, cloud, and more, with 70% of those incidents dealing with three or more of those fronts, right? And so that is the key data point that really drives a need for SOC transformation to have visibility into all those different environments and fronts and be able to stitch that data together to be able to tell a story, right? Just because you're bringing in the data, that's step one, but you need to be able to have a system stitch the data together to be able to tell, okay, the threat actors started at the human layer, right, whether phishing or whatever front that may be, move to the endpoint, then to the cloud, and exfiltrate our data, and you want that all within one view for the analyst to be able to very quickly understand what occurred and be able to move from there.
David Moulton: Clay, earlier you mentioned artificial intelligence, and I feel like this term has just become the norm. It's not the new. It's not the novel. It is. But I want to talk about it because I think it's important, and I'm going to pair that up with automation. You know, AI and automation are often seen as the future for SOCS. How do these technologies enable SOCS to stay ahead of some of the threats that we're seeing and talking about today?
Clay Brothers: Yeah, that's a great question. So I've had the fortune of working really closely with our Palo Alto Network's internal SOC to be able to learn how they do things, which I believe is best in class, and oftentimes when our customers are looking at our SOC, they say, "Wow, this is great. Like, this is the future. This is where I want to move towards." So with AI machine learning and automation, so let's start with AI machine learning, I think it's really critical to have a system in place that can bring all the data together, stitch it together, normalize it in a very quick fashion. Typically, this takes months of a project just to get the data in and normalize it so that it's comparative against the other data sets. Now this can be done in hours or days, right? From there, right, you want to be able to detect attacks, right? Having detections in place, you know, unique to your business that maybe you're engineering is always going to be important, but if you have the ability to rely on AI machine learning as a foundation to be able to detect anomalous activity, suspicious or malicious activity that seems outside of the baseline of the norm for a user or an application or a service, that is the way to truly detect attacks today, right? You know, these threat actors are also using AI and machine learning, and so once again, going back to that legacy SOC concept, relying on static signatures to be able to notify your SOC that something's bad happening is a way of the past and is not going to work anymore. You know, in these SOC transformation projects I work on, we very quickly understand what tasks the analysts are taking the most time on, and those are the candidates for the use of AI or the use of automation to either replace or supplement the analyst's time. Our SOC measures how much time they believe they've saved with automation and AI, and right now that's at 65 full-time employees, right? And that is driven by their analysis of what tasks take the most amount of time and monitoring that over time to be able to understand how many times that action's taken, you know, multiplying that by the time that you typically required to do that manually, right, and they're able to manage that, once again, the offsetting of 65 full-time employees.
David Moulton: That's pretty amazing, because there is such a lack of skill or capacity for specific skill and such demand for those folks that are around that you would want to say, like, how do you address the, you know, I think over the last 10 years as I've been in this industry, it's always like two and a half, three and a half million open jobs. There isn't a way to just conjure up millions of people to do this work, and it becomes more and more important, as you were saying earlier, with speed, scale, sophistication, with the advent of really disruptive attacks that are shutting down entire businesses or systems. Where does that answer come from? So maybe the answer is, well, we do lose some of those lower-level skills and deep, deep understanding. We give ourselves time to go to a higher level of management of things and you develop a new craft at that next level on a foundation of AI automation that you can monitor, and really one of the things that I keep hearing and I've started to see myself is the AI that you're working with today is the worst version of that AI. I don't know when that will slow down or when we'll say, like, "God, no, we need to, like, roll it back to version, you know, 64," right? But it seems to me that each time you come back a week, a month later, you're going, "This is far more powerful, far more capable than it was just a week ago, just a month ago." If you go back a year, you're going, "It's hard to comprehend how much better it got in a year." And so that foundation, I think, continues to get better for you, which pushes you as the operator, as the SOC analyst, you know, the security provider, further and further into your mission.
Clay Brothers: Yeah, I mean, look, no one wants, no analyst wants to spend their, you know, eight hours a day responding to alerts that end up being false positives, right? Which is the very traditional normal way of life, is that the majority of alerts are false positives. There's no action needed. It just was time taken from the analyst to review it, analyze it, and make that determination, right? That is providing very little value to the organization, right? AI and machine learning, I agree, may not be at its -- or definitely is not at its peak, right, but it definitely can take those tasks out of the hands of analysts so that analysts can do more value add, but more fun things, right? Like, you know, there's also -- SOC analysts, you know, have a short lifetime, typically, right? They're in the career for two or three years and then they move to another job, right? It's a churned job and -- but I think we're changing the game here where they can focus on, quote/unquote, cooler things about cyber, right? Like threat hunting or continuous improvement and tuning and detection and automation and things like that, where the analysts can be focusing their time there, right, while the autonomous, you know, self-driving SOC, it's doing its thing below them, they can focus on other things, which I think is good for the industry, good for the people, and I do think it's driving better security outcomes as well.
David Moulton: When you were talking about that, I went back to conversations with the folks in our SOC and the tenure that they have is incredible, right? And I think it goes to this idea that your job day-to-day has very meaningful work because you're looking at 10, maybe 12 overall events that you've got to investigate, it's not hundreds of thousands, and it's very, very likely that when you're running down one of those, you know, 10 events in a given day, that's significant for the business. It actually was a security incident with a meaningful amount of work that you did to protect the company, rather than you closed out page after page of tickets and they were all just nothing, right? Who wants to do that? Let's switch gears a little bit and talk about threat detection and response. What are some of the most significant changes in threat detection and response strategies that modern SOCs must adopt to?
Clay Brothers: Yeah, so in the past, it was all about creating manual detections, static signature-based detections that meet a certain criteria that if that criteria is met or that threshold's met, then it would generate an alert. Today it's all about cloud native-based detection. So our Cortex XIM solution has 100 threat researchers that are behind the solution, that for every single customer, they are constantly tuning and adding new detections that all of our customers can take advantage of because it's on the cloud, right? It's that SaaS-based product. That is going to allow customers to spend less time understanding the development language and creating those queries and those detections and more time responding to them, right? In every case, in every customer I have, you're always going to have unique business requirements that require you to create those detections, which is why it's really important to have engineers either in your SOC or outside of your SOC that are working really closely to develop those, but you can rely a lot less on those folks responding to those attacks because you have these native, out-of-the-box detections that are constantly being optimized for your business. To me, that is the way forward, is to be able to rely on experts to help supplement your team in actually detecting those needles in a haystack.
David Moulton: Talk to me about how Unit 42 integrates threat intelligence into these strategies.
Clay Brothers: Yeah, so Unit 42, you know, has a model called, or a tool called "TIKR," Threat Intelligence Knowledge Repository. I believe that's what that stands for.
David Moulton: Love me a good acronym.
Clay Brothers: It is kind of the crux of Palo Alto that is pulling and receiving data from our Palo Alto products, right? The millions of firewalls and millions of endpoints, the hundreds of thousands of cloud instances and more, right? And pulling and pushing that data set, but then also from our instant response and proactive services and our MDR team and taking data from them and providing data back. We are seeing and collecting all this data and being able to make sense of it, make decisions, understand trends, as well as our own independent threat research and deep dark web analysis that we can leverage to be able to push data to our products and to our services so that we're better serving our customers. [ Music ]
David Moulton: Clay, what metrics should SOC leaders be focused on to measure success of these transformations?
Clay Brothers: So this one is near and dear to my heart because, you know, back, I'd say, six, six-plus years ago, you know, it was, you go to an industry event and it was, you were the best SOC leader if you said, you know, one SOC leader says, "Hey, I've got, you know, 150 alerts a day," and the next SOC leader says, "Oh, really? I have 430," and the next one's got, you know, it was almost better if you had more, right? And it's changed now, you know, where you're actually driving for less. But in terms of metrics, it's really hard, right? Because in SOC, in the SOC world, metrics can be gamified, which is not always a good thing, right? So like the classic mean time to detect, mean time to respond, mean time to close, right? If SOC analysts know that one of the key metrics that they individually are graded on and their performance is tied to is how quickly you analyze and close a ticket, you can imagine the kind of behavior they may take to close that preemptively or as quick as possible, right, which is usually not a good thing, right? Usually you want to take the time and make sure you're really analyzing the data and the events that are provided to you as the analyst and make informed decisions. But that being said, assuming you're able to manage that properly, meantime detect, respond, and close will always be a key metric. Oftentimes with our SOC transformation customers, we are seeing a major improvement in those metrics in terms of how quickly they're able to detect an alert on an attack, how quickly they're able to acknowledge it and start working on it, remediating it, and closing it, right? Those are obviously the whole lifecycle of a SOC analyst, but there's more, right? In terms of automation and how many of tasks or FTE offset you're able to take on, how much time you're able to spend on threat hunting and more value-add activities versus, you know, manually responding or creating new detections, right, those are typically the key metrics we see. That really drives, you know, your head count, your structure, your budget, you know, and there's a variety of other metrics out there, but, you know, it's -- I know most SOC leaders and managers know this, but it is so critical to make sure you are managing those metrics clearly so you're not driving bad behavior.
David Moulton: So SOC transformation, as I understand it, is this balance between people, process, and technology. How should organizations align those three things effectively?
Clay Brothers: Yeah, they're all important, right? I mean, technology is going to be, you know, the heartbeat of the SOC, right? Having a strong SIM tool that can use automation, use AI and machine learning to help make the lives of the analysts easier, stitching the data so you have full visibility into the environment, right, all of that is critical, and that's, to me, that's like step zero, right? You have to have that in order to be successful. Now, you can have the shiny new toy and still fail, right? So having the right people in place that have that instant response experience and mentality is critical. Knowing how to leverage the technology is critical. Not every -- you know, moving from one SIM to the other is not apples to apples all the time. And then process, right? SOC transformation focuses a lot on enabling the SOC from a people and process perspective to leverage and optimize the use of the technology, right? And if you don't have all those different pieces in place, it's not going to be successful. You're going to have incidents that are occurring that you may not be aware of. You're going to respond to attacks inefficiently, right? There's going to be a lot of problems that could cause massive disruption to your business.
David Moulton: Resilience is a key theme in cybersecurity. How does SOCs build resilience against the sophisticated threats that you mentioned earlier and the supply chain attacks that we're seeing against the disruptive behaviors used to gain leverage by threat actors? What is their role in building resilience?
Clay Brothers: Yeah, you know, resilience has a very vague definition depending on who you're talking to. You know, my definition of resilience is being able to very quickly bounce back from attacks, right? The SOC is literally the front lines of that, right? They are going to be the ones that are going to be able to detect and respond to those attacks. Now, prevention is step one, right? One of our SOC's key principles is prevention first, detection second, right? And that, I believe, should be the mentality for everyone, right? If you can prevent it first, then that should be the number one priority. The SOC oftentimes sees prevention as another team's responsibility, but the SOC, based on what they're seeing, they can be the ones that can provide recommendations and guidance and prioritization of what should be prevented, right? They are the ones that can identify, hey, we're constantly seeing these administrators use PsExec to remote into various systems, right? We know that this is a common way in which threat actors are moving laterally. Can we have these administrators use a different path, or is there a way for us to very quickly understand what's the authorized use of those, right? And so the SOC being able to influence the prevention and best practices is, to me, that's helping with that first step of prevention. Constantly optimizing that detect and respond is that second line, but then three, the SOC, if there is a major incident, SOC resilience and instant response go hand in hand. Being able to very quickly understand what the threat actor did, what they got access to, how to get them out as quickly as possible, right, that is not only going to be the SOC's job, there's going to be a lot of other individuals and teams involved, but the SOC needs to be upfront and personal with that. You know, oftentimes, you know, these SOCs are partnering with firms like us to help with that digital forensics and instant response and getting those threat actors out as quickly as possible, and that's definitely a best practice, is leveraging, you know, the retainers and relationships that you have to be able to drive strong cyber resilience.
David Moulton: Where does the work you do and you lead play a role in our clients' resilience?
Clay Brothers: Yeah, usually it's, I mean, ideally, customers are coming to us before a major incident and seeing, you know, the speed, sophistication, scale of these threat actors and understand they need to make a change to their SOC. Other times there is a major incident and they realize that there's gaps with their SOC and they need expertise to help put together a roadmap of how to get from where they are today to where they need to be in the future, and that future state may be aligned to industry best practices. It may be, you know, they want to emulate what our SOC is doing and more, but that is -- our SOC assessment is all about understanding where the customer is today, right, people, process, and technology, and understanding where is their North Star that they're trying to get to and putting together a roadmap of prioritized recommendations on how to get there, and then partnering with them as part of SOC transformation to help them get to that goal.
David Moulton: Do you have a success story where Unit 42 really helped a client transform their SOC and improve their security outcomes that you'd like to share?
Clay Brothers: Yes. Yeah. So I mentioned this before, but one of my major customers when I first started here was a major oil and gas company. We did a SOC transformation project with them where we started from the beginning, right? We were building SOC charters and instant response plans on defining exactly how the SOC should be structured, how they should, you know, be staffed and organized. We helped them provide a recommendation on the SIM tool they should move towards. We did some vendor discussions. We helped them based on their business and their use cases and the log sources they had. We helped with metrics, generating metrics and, you know, building what that reporting structure should look like, and more, right? And that was easily my favorite story because it was, from beginning to end, we touched all things SOC. Now, I would say the most exciting story I had was more of a tactical story where we had a customer where they hired us to just help them with developing some instant response plans. They wanted step-by-step instructions on, hey, what happens if we're hit with ransomware? What happens if we're hit with business email compromise? And a couple others on their list. As we're developing those, one of them that we were wrapping up was insider threat, and sure enough, it was like a Friday afternoon, and we get a call from SOC saying, "Hey, we're not," you know, "We appreciate you helping us build this. I know we're still in draft form, but we need to enact this, like, today."
David Moulton: Yeah.
Clay Brothers: And it was really unique because it was an IT administrator that they believed was exhibiting some suspicious behavior that had the keys to the kingdom, knew all the vulnerabilities that existed at the organization, and was just a massive threat to the organization if they were to be an insider threat. And so we built a playbook that described, all right, here's things you need to do before you notify and make this employee aware that, you know, we're on to him. Here's the things we need to do if the conversation goes well, right, it was a misunderstanding. Here's what we need to do if the conversation goes south, and they realized he was caught, and, you know, the SOC was a major part of that, but it also, of course, involved legal and the executive team and more. To me, that was being on the proactive side. I do have some battle bruises from instant response, but it's not the day -- it's not the day in/day out I live in, but being involved with something like that, you know, it's very exciting to be able to help organizations, you know, build a plan so that they feel good about what they're doing and rather than just kind of hacking at a, you know, a tree to be able to understand, you know, what is it they need to be doing.
David Moulton: So we've covered a lot of ground here and I want to take us into the future. You know, as these threats all evolve and organizations are really trying to future-proof their SOCs, what do they need to do to remain effective and relevant, say, in like 5, maybe 10 years down the line?
Clay Brothers: Yeah, I mean, our vision is the autonomous SOC.
David Moulton: Yup.
Clay Brothers: You need to move to AI machine learning and automation to help drive your autonomous SOC similar to, you know, an autonomous car, right, a self-driving car, because that is the way of the future. The speed, sophistication, and scale is going to continuously increase. It's going to be harder and harder for the SOCs to detect threats in that legacy environment, and you have to rely on this new model in order to be successful. So getting there now is going to help you in the future, versus waiting, you're going to get further and further behind. You know, knock on wood, I hope this doesn't happen to any of my customers out there, but, you know, if you stay in that legacy environment, you have an increased likelihood and impact of a major security incident disrupting your business, right? Moving to this autonomous SOC I think is critical for the business in achieving its mission and protecting the data that it has, but it also is critical for your analysts. This is the way -- this is where analysts want to go. This is where they want -- the type of SOC they want to work in and it's the right SOC to work in.
David Moulton: Clay, thanks for a great conversation today. I really enjoyed learning from you as we talked about SOC transformation and getting into a little bit of your history as a consultant at UI [phonetic], all the way to the proactive work and the transformation work that you lead here at Palo Alto Networks.
Clay Brothers: I appreciate being here. It was a lot of fun, and it was good chatting with you, as always. [ Music ]
David Moulton: That's it for today. If you like what you heard, please subscribe wherever you're listening, and leave us a review on Apple Podcasts or Spotify. Those reviews really do help us understand what you want to hear about. If you want to reach out to me directly about the show, email me at threatvector @paloaltonetworks.com. I want to thank our executive producer, Michael Heller, our content and production teams, which include Kenne Miller, Joe Bettencourt, and Virginia Tran. Elliott Peltzman edits the show and mixes the audio. We'll be back next week. Until then, stay secure, stay vigilant. Goodbye for now. [ Music ]
