From Identity Compromise to AI Defense: Inside Modern Incident Response

Elliot Volkman: Hello, and welcome back to the "Microsoft Threat Intelligence Podcast". Today I'm going to be bringing you two segments that we captured at Black Hat USA. It's going to technically give us two sides of an interesting security equation. On one side, we're going to be looking at folks from Microsoft Incident Response or DART, and some of our folks from the Defender research side or intelligence side. One conversation digs into identity compromise and what that generally means during containment. The other is how a Purple team, or our Purple team in particular, recreates emerging attacks, including AI-driven attacks, to find the gaps before our threat actors do. Together, these will show how the attack surface is changing and how security teams are adapting. Alright, on to our DART crew. Welcome back to the "Microsoft Threat Intelligence Podcast". We are filming here at Black Hat 2026, and fortunately, well, we're going to skip through the pleasantries because we have a short amount of time with some very important people. We have Adrian Hill and Terry Mee of our DART, slash, what we actually call our incident response team. Today, we're going to be covering something of a critical nature in this world of AI that we're living in, and that's about how we respond to incidents primarily tied to identity. But Adrian, maybe we can start with you. Give me a little bit of a background on yourself here so we can give some context.

Adrian Hill: Understood. So my name is Adrian Hill. I'm a lead investigator with Microsoft's Detection and Response Team. And I spent 21 and a half years in the United States Marine Corps, primarily in defense of cyberspace operations. I transitioned over to Microsoft at the midway point of 2021 after I retired and then came over as a lead investigator. And that's my background in the cyber defense space. 

Elliot Volkman: Love it. Terry, how about yourself? 

Terry Mee: Yeah, again, another lead investigator, military experience as well. Thirteen years in the British Army, again doing defense of cyber operations, and then I've been doing instant response for, in the public sector, for the past five years. 

Elliot Volkman: Appreciate that. All right, thank you for that background context. I think it helps set up the scene for what we're going to be talking about here, which is a slice of the piece of the equation that you already walked through because there are endless attributes that come into an incident. But let's go through the identity piece of the equation, which is when you join an identity incident, and what are the first signs that this is bigger than a single account compromise? 

Adrian Hill: I'll take the lead on it. When you look at identity, it is the foundation of which all things live and exist in the world of security. It ties to your users, your administrators, your applications, devices, everything has an identity. And so if it's just a single account that was compromised, what tells you it's a bigger problem is, where does that identity live? Is it in the cloud? Is it on-prem? Is the identity tied to an application that maybe has a lot of permissions? It's overly permissive application. Is it an administrator account? Is it an administrator account in the cloud? You know, is it a global administrator, is it a security administrator? Is it an application administrator? Those signals will tell you, you know, is this incident just it's a regular user that got compromised? We can simply reset the password, maybe disable the account and investigate further. But when you look at those details, it tells you a lot about how big or expansive this thing could get and how much time you really do have to respond to it. Terry? 

Terry Mee: Yeah, I agree. Again, you know, what the identity has permissions for, but also trying to understand what was the kind of the root cause of the compromise, because we go into a lot of investigations where it's a single identity at first, and then we find out it's actually a bit more expansive and more identities have been compromised. So it's really important not just to look at what that identity has access to and missions and controls and what may have happened during the time the threat actor had access to identity, but understanding, you know, where else that threat actor might have got access to. 

Elliot Volkman: Perfect. That's actually a really good segue into containment. But I've got to ask, so it is generally probably known if you're a part of the equation in this situation, there is a known incident at this point. But in some cases, probably, or earlier in the stages, there's probably what appears to be a legitimate use of credentials and trajectory. But how do we look through the lens of containment? Maybe before it gets into your hands. Obviously, now it's an active incident. But how are we looking at signals and information if it looks like legitimate traffic, and then how do we shift to containment if we're realizing, oh, this is not smelling right? 

Terry Mee: Sure. So when we say it looks like legitimate activity, it may be a case of, you know, a threat actor is using IP addresses from a similar location [inaudible 00:04:47] where a normal user might log into, trying to blend in, or being very discreet and quiet, so only doing a little bit of activity every now and then. So that's where it's really important to have some kind of behavioral analytics to help you detect those issues, and also having like a breadth look across your whole identity space and understanding what does normal user interaction look like, or interaction to identities, and what does abnormal look like. So having a good baseline and a foundation and understanding what is normal in your environment really helps defenders to pick out stuff that, you know, might not get detected or might kind of look suspicious, and they can get a good understanding and then figure out what's going on. Because they, the customer or the organization, always knows their environment better than we do, so they're the ones that ideally would be in a better place to see, you know, what looks unusual for their environment. 

Adrian Hill: Yeah. And additionally in that the -- knowing your baseline, but then also ensuring that you're logging that information, because when we come in from a containment perspective, we prelead the data and we need to know where it lives, and hopefully the customers know where that data lives, and that they've -- they're logging the right things. You know, sometimes you may only be logging like, you know, a third of what you actually need, or you might be logging everything, and of course if everything's a priority, nothing's a priority. But I think that's where it stems from, to his point, knowing the baseline and ensuring that you're keeping track of -- that you have the correct telemetry coming in and that you're retaining it for the appropriate amount of time. Yeah, that's the way I look at it. 

Elliot Volkman: That makes sense. And maybe we can drill down to it like a tactical example, which would be like MFA. So, on your side, whether you're logging or not, it might look like it's appearing to function correctly, but a threat actor can still just slide through. How would that function, where they're just sneaking past it? 

Adrian Hill: It depends. I mean, threat actors do a lot of sneaky different things. So if there's a man-in-the-middle attack or if there's infrastructure in the middle where they're capturing credentials, so the MFA looks good to you if it was successful, but they may have captured a session and they're replaying that and using that to get back into the environment. That's one method of it. A lot of times, users in of themselves, they receive that MFA notification, and then they, you know, if they click that button, even if they're not paying attention to the fact that I'm not the one who've -- I'm not the one who generated this, but sometimes people, there's the fog of information technology, and so they'll just click it, and it wasn't even you who did it. So someone captured those credentials and they could use, potentially, use those later. So MFA is good, but it's how you use it. Are your employees, are they able to report if they're receiving erroneous or false requests to use MFA? Do they have the ability to report that up to someone and heighten it to other levels? 

Terry Mee: Yeah, I don't -- just say that not all MFAs are created equal. So we've got, in Microsoft, we've got the 'good', 'better', and 'best'. And we always want to try and get to 'best', especially for our highly privileged accounts. And ideally, we want to try and get our users to there as well, but if we can't, then get into 'better'. So that's kind of where we see some issues with MFA, where if organizations are still using SMS. So we do have threat actors, one of the most notorious ones at the moment is Octo Tempest, where they use SMS phishing or they port known phone numbers across to their own control and then they're able to authenticate the MFA themselves. And so we really want to try and push the boundary now to getting to that 'best' kind of MFA, you know, and we need to keep iterating and keeping up to date with all our security technologies, making sure we're going for what is the best options for us, because these threat actors will just find ways to exploit them. 

Elliot Volkman: Mm. That makes sense to me, so I was going to ask the question, and maybe we can clarify just for the sake of who our audience is, but when we're talking about 'good', 'better', 'best', it's not necessarily a technological thing, it's probably process, implementation, and the controls and policies mostly, is that's probably like the heaviest part of it? 

Adrian Hill: Passwordless. 

Elliot Volkman: There you go. Yeah. 

Adrian Hill: You know, if you can move into passwordless MFA, the legacy -- it's not called legacy, but I mean to us it is, there's the SMS or the voice calling. Do whatever you can to remove yourselves from using those systems and go to passwordless, obviously using like a FIDO2 key, those things are really where that lives at, and especially for your administrators, or if there's something that you want to put behind that paywall of security, moving towards not just passwordless, but going all the way up to the FIDO2, is that's where you want to be. And then again, conditional access policies as well, right? 

Elliot Volkman: Yep. 

Adrian Hill: If you want to move to phishing-resistant MFA and things of that nature, you want to put in the best, the most advanced technology that you can. And we understand there are use cases out there where customers say we can't implement this, because maybe they have a program or record or some legacy function behind that. But where they can do it, they should. 

Elliot Volkman: Okay, that makes sense. So I want to pull back on something that you had mentioned before, which was logging and accessibility information, what organizations will hopefully have versus not. If there is maybe one thing that should be tracked and logged and captured in the sake of, unfortunately, if their worst day comes and there's an incident, is there one thing that organizations typically don't but should start logging today? 

Adrian Hill: I wouldn't necessarily say you could peg it down to one thing, but essentially if they should just have centralized longing in the environment, period. Because that, sometimes we'll come in and they don't have -- like, for instance, there'll be a section of their environment where they were trying to modernize it, but the plan has been on stall for whatever reason, and so it just gets lost. And there you have a lot of shadow IT and things where they're not covering down on what should be getting passed into their scene or whatnot, and maybe they're switching between appliances or between services, and there's a lot that can get lost in there, but I think if they do the right business impact assessment and just take a look at what's most important, then they'll figure that out pretty quickly. Because the one thing you do not want is for there to be a shadow area where nothing is being captured. We come into that situation and, for the most part, we can't do much if the data doesn't exist. Terry? 

Terry Mee: Yeah, I'll just say retention is the big one. Like, if you're going to pick one thing, is making sure your retention policies are long enough. If you're just having a default retention policy, for instance, enter ID logins, you know, 30 days and you not recall anything back, then it makes it quite difficult for us to go back in time and work out when that initial compromise actually happened. So retention, if you're going to pick one thing, I wouldn't pick a specific log, I'd pick your retention period and making sure it's long enough you can go back. So it used to be the industry standard was around three months, but now sometimes we're seeing, especially with nation states and their environments, quite a long time, so you kind of want to be pushing back for those critical logs at least six or 12 months. 

Elliot Volkman: Okay. 

Adrian Hill: And I'd even throw in there that, you know, you look at some organization or some industries like banking, you know, they have regulatory requirements where they may have to retain records for three years, or retain logs, records, for three years. Some are it's a year and a half, two years. And if, to be honest, in incidents where you look back at the day, it's like I wish they had a policy that was like that, where they could have maintained it for a longer period of time. And I get it that consumption, for some, is an issue. But we found that some customers that are ahead of the game, they'll retain the logs and then they just -- the data goes into cold storage, and if we need it, you know, they could -- it's a process to get it turned back on, but it has helped us in some of those long-term investigations where the threat actor may have been in the environment for years at a time, or at least preceding what they would normally retain for, and they're like, let's go back and get those logs turned back on so you can access them. And that, that's a big help. 

Elliot Volkman: That makes sense to me. So I think, now we're sniffing through the logs, we're identifying something suspicious going on, is there a moment in time where you make the decision to revoke access, to force rotating keys and -- basically a scenario that you and I have talked about in the past, which is during an incident, the temperatures are very high, your roles are really to come in and help bring down, but that is a mechanism that could create a little bit of disruption and chaos. So how do you make that decision on when to basically pull access and basically add disruption? 

Adrian Hill: It depends. It's, number one, it's always a business decision when we're going to do something that is as drastic as that. But for the most part, it's getting the message across of, what happens if we don't do this? And when we have -- you know, you can't unring a bell. And so thus, in these situations, it's most of, can we explain to the stakeholders, to the appropriate folks that have the authority, what is the outset of this that's going to happen if we don't do it? That's really what it comes down to, the brass tacks of it. But mostly it's going to be if we know the threat actor is in the environment, whether they're destructive or not, because sometimes you have to collect the data and you need to see what's going on before you can make certain judgments, but it's -- that might be the tell of it. Is it destructive? If it's not, at this point right now, let's see if we can just isolate the environment before we start doing mass password resets and things of that nature. And you might not have all the data at that time where you think, it's a go, we should do this. And if you -- and you won't always have perfect data. There's no such thing as it, to be honest. So each situation is different in how you deal with it. But if things are starting to get to the point where it may look destructive, or we feel like the spread could get out of -- to a point of no return, that's when we're going to make that decision and explain that to the folks who have the authority to turn it off. 

Terry Mee: Yeah, so it's basically risk, right? Risk management. And what helps you with that is understanding who the threat actor is. So having more data allows us to build a better intelligence picture and kind of narrow it down to who it is, and then we can kind of understand what their motives may be, and then we can advise customers, you know, we know this threat actor does X, if we don't cut them off then, you know, it could be, like Adrian said, be disruptive, they could look to steal data, they could look to do something else. So understanding what their motives are really does help. And also, once we understand what their motives are, we need to then think about, okay, if we know they're going to be destructive, do we believe we've got all of the identities they've got access to. So if we only take out, you know, half of them, or we leave one, then that threat actor might just switch straight to destructive. So there's -- you've got to make a choice then of, do we wait to collect more data, or do we just have to take that decision to cut off the network and then, as Adrian said, start doing mass resets just to be doubly safe that, you know, we're not going to let that threat actor come back in. 

Elliot Volkman: So, a key word that jumped out there to me is looking at this through the lens of risk, which is a good little pivot over to our ongoing and exceedingly painful supply chain attacks, which are a little bit of a unique situation for most organizations because, in a lot of cases, it's generally outside of their control. Like, we have third-party risk management, TPRM, and it goes through GRC, but like, that is not going to prevent a downstream situation or curve. So that begs the question of, where do organizations underestimate maybe SaaS and OAuth for systems and things that could also result in a supply chain attack? 

Terry Mee: Yeah. So we're looking at SaaS and OAuth. Like a standard containment procedure for an identity might be to just reset passwords. So some customers may think, oh, we can just reset the password and then the threat actor won't have access to the account. You've got to think about session tokens. So when it comes to application OAuths, the threat actor's going to have a session token for that, or they can have another API key that they've gotten from that identity. So you've got to think about all the ways that potentially compromised identity or the application can authenticate, and those have all got to be revoked and changed as well. So it's not just as, like the good old days, where you could just reset a password of and disable an account and then get the user to put in a new password. There's a lot of extra things we've got to think about now around identity. 

Elliot Volkman: Anything you want to add, Adrian? 

Adrian Hill: Yeah, you don't want to get into the game of playing whack-a-mole, and that's typically what it becomes. If you take a half measure, then you will end up right back where you were in the first place. So you have to make sure that you look at the whole field and see what all, like you said, is it the session, is it the MFA device? Because maybe the threat actor registered a new one. What are all the areas of access that this identity has access to? And obviously, if this is a credentialed and a privileged account, in that case you have to go much further. But if it's a regular user, then in those cases it's a lot easier to just say we can reset, revoke all the sessions. If there was MFA associated then, you know, maybe disable and re-enable, so on and so forth. But that's really what it comes down to. If it's a privileged account, in that case, you have to take it all the way, you know, take it all the way home. 

Elliot Volkman: Cool. So, with that being said, not all identities are obviously created equal. Agents and AI. A little bit different scenario, but honestly, what's old is new again, elevated access. So I'm curious how you all differentiate and look through that lens of the different levels and impacts of identity, whether we're looking at AI agents, or even back to, you know, SaaS technology, where they have more persistent access. 

Adrian Hill: Going to take it? 

Terry Mee: Yep, sure. So, with identity and everything, we always look at resetting permissions and stuff like that, and with, you know, with Microsoft we've got the -- we have the tiering model, and now we've got the enterprise access security model. So we're kind of looking at the different layers and what those accounts may have access to. So you've got your normal users, as Adrian said, if it's normal user, then we're kind of looking at what permissions they might have. And, I don't know if that's going to be picked up in the recording. I'll carry on. So yeah, we have the normal user accounts, like Adrian said, and then we've got like our service kind of accounts, which they may have quite high privileges. So we see a lot with the service accounts where, I know you just mentioned third-party risk, where other software applications come in and they're just like, oh yeah, we need a service account and just give us domain admin, just give us global admin, because then everything just works. And we don't really want to be there. We want to be able to restrict down to the least privileged as possible. And then just make sure that, you know, we're logging and everything like that properly there, and then we've got the privilege accounts, highly privileged accounts, and as you -- as Adrian said, we've got to make sure that we -- you take everything away. And it's the same with, you know, if you've got AI agents and stuff like that, then they'll have some kind of approved access, whether you've given them a service account or API keys or whatever it is to authenticate, then we've got to just treat them as we did any other identity and understand what permissions or what access that has access to, and then just see if any of that data or any of that access has been leveraged. 

Adrian Hill: And to key in on a point that Terry made there about the privileges that you give something and when you talk about AI agents is, I think, in a lot of situations, when we go out and we see that there's overprivileged accounts, whether it's an application or a service account, and we ask, well, you know, why is that, when the organization, the company, or the customer goes back and asks, like, how did this get to this point, it was, we needed it at the time. And but there's not a real clear picture on why this service account needed to have these privileges, or why a service account is being used for interactive logons, and so on and so forth. And in those cases, it really comes down to, what is your policy setup before you even enable agents in the first place? And are you tracking what agents can or cannot do? And are you restricting folks' ability to use agents in your environment? That's, you know, before you even get to the fact that it has an identity and privileges associated with it, what's the policy that starts that off? Because what we're finding as we go out and investigate is there's a lot of agents being used, there's a lot of AI tools being used in the environment, and folks don't know where to capture that data, but luckily Microsoft has the tools to help you see that picture and rein in those controls, or put the controls you need necessarily in place. 

Elliot Volkman: That makes sense to me. So, considering our audience is obviously practitioner, analysts, researchers, folks like yourself, I'm curious, how would you help position this in context so that they can bark up the chain, hopefully not to like the level of board of directors, but explain the attack surface that comes with not being cognizant of scenarios exactly like you're calling out, where there's not a lot of rhyme or reason, and maybe it was just in the moment they needed access, but they should have restricted that back down? 

Adrian Hill: I feel like -- or, at least, not feel, but in my experience, what often comes to when we ask these questions of how things got to this place, how it got here in the first place, was there wasn't really a policy that was driving it. 

Elliot Volkman: Okay. 

Adrian Hill: And, you know, folks will, when they want something done, you need to build an application that does XYZ, they're just like, let's just build it and get it done with. And the security is an afterthought for most people to fall outside of the security space. And for the practitioners who are struggling to rein these things in, I think where it starts from is they have to talk to their leadership and say, hey, can we take a -- can we look back at the foundational policies that are in place? And like, what is the company's going position, or at least the security folks' position on this, and are we talking to the right partners inside, internally, to make sure that we're all on the same page? To give you an example, when there's firings, which happens, right, is there a conversation between HR when someone's account has to go? And it's not even just a firing, it's someone's leaving the company, they're seeking employment elsewhere, doesn't matter. But when they do that, is there a conversation with HR for what happens to this account, and if that account had privileges associated with it, were they disabled, was it removed? In some cases, that conversation never happens. And so you might have an active account that's in an environment that's just there and this person left five years ago. It's like, there should have been a policy there and a conversation between some folks to ensure that those types of things don't happen. But I think it just comes back to first principles. What's the policy? What's driving it? And if there's something that is up in opposition to why you would want to put those policies in place, ensure you have the right stakeholders there, explain the risk, and then once they get on the same page, I'm sure they'll see it from where you do. Whether or not they're, you know, happy to oblige you, you know, that is what it is. 

Elliot Volkman: Anything you want to add on? 

Terry Mee: I concur. 

Elliot Volkman: Is it documented  and hope they listen? 

Terry Mee: [laughs] Policy, you've got to check everything, right? So on a yearly basis or a quarterly basis, depending on what kind of your process is, you want to make sure that you're checking what you think is set, is set. So that whether that's with identity, whether it's with configurations, whether it's with access, you want to have a review on a regular basis to kind of understand that, okay, is this a situation, you know, we reconfigured this last year, some changes may have happened over the year, is it still according to policy or have things been changed to just make things work [inaudible 00:24:27] time? 

Elliot Volkman: Cool. So we're going to leave us on a slightly different note to kind of like take us home, but a lot of the work that you do is, again, on our customers, is your focus worst day more or less. To prevent that from happening, prevention or proactive exercises, is there some sort of tabletop exercises that you can point people towards that'll help reduce some of this blowback on identity-based risks? Adrian Hill: Indeed. I mean, Microsoft Detect and Response Team has tabletop exercises or cyber readiness exercises, as well as executive tabletops and a cyber range, which we actually have here today at Black Hat. So we will be putting on a cyber range for folks that goes at 12:30, I believe is the time, if I'm off on that. I'm selling it right now or trying to. There's also, when you meet the experts, there's a mini CTF that you can engage in over there as well. But we have what you need, come and have a good time with us and let us help you to prepare to get on the right foot before the worst day comes down, because it won't because you were prepared. 

Terry Mee: And just to add, we do offer identity assessments and compromise assessments and identity hardening as well. So if you need support on them, in getting those implementations, getting those controls put in place to help protect your identity, we can also provide that. 

Elliot Volkman: Cool. 

Adrian Hill: Indeed. 

Elliot Volkman: So, if I hear that correctly, we should be investing time wisely in proactive measures to make sure we don't get to the state where you're called in. 

Adrian Hill: 100%. Indeed. It's -- the day to learn is not the day that it happens to you, it's to try and be as prepared as possible. You know, no plan survives first contact, of course, but if you were doing your due diligence to get on the right page before it happens, you'll be in a much better position to respond at a speedy scale. 

Elliot Volkman: All right. So this has been another episode of the "Microsoft Threat Intelligence Podcast". Adrian, Terry, thank you for joining us. We'll be back with you next time. Welcome back to the "Microsoft Threat Intelligence Podcast". We are filming live or live-ish, you're listening in on a recorded version of this, at Black Hat 2026. And we are chatting with some folks on our Purple team organization who help to counter attacks that happen across our ecosystem. Is that a safe way to say it or I'm still going to get in trouble? 

Matt Burrough: No, I think that's a safe way to say it. 

Elliot Volkman: Okay, I'm good with that. So I think one of the interesting things that we look at when we're talking about pen testing, Red teaming in the world that we're in, the world of AI, a lot of organizations think about like prompt injections and trying to get it to behave a certain way. I'd like to flip that script, because I feel like that is not what you all focus on when we're trying to break technology and make it more secure. So maybe we can start with this. Matt, help us understand what your team does. How are you breaking stuff and making it more secure? 

Matt Burrough: Yeah, absolutely. So we are the Defender Purple team. We exist within the Defender organization to make sure that the product is catching all the latest attacks, detecting them, alerting on them, have proper alerts, proper blocking, all that kind of work. So we actually work as a Blue team and a Red team together as one, along with infrastructure engineers and AI engineers. And so we'll research what the latest threat actors are doing, we'll build out a simulated attack, we'll build out a simulated lab environment that looks like what our customer environments tend to look like, and then we'll stage that attack. Does your defender find anything that needs to be fixed? Get that in the product and run it again and make sure we're continually improving. 

Elliot Volkman: All right. Jorge, anything else you want to add? Some additional color and context? 

Jorge Calderon: Yeah, it's an interesting way to do testing because there's a lot of telemetry that you don't really see when you're just doing two TPs one at a time. So we kind of specialize in doing the entire attack chain. So you read something in the news, the more of that you can recreate in a simulation, the more opportunities you give to your EDRs to find opportunities for disruption detection, things like that. So it's a really comprehensive way to kind of vet how the security products are doing. 

Elliot Volkman: Excellent. So I'm going to bridge a conversation that we had just captured, which was with folks from our DART, or Incident Response team, the lead investigators. We were talking about identities and protecting identities, some of the challenges associated with it. But we're also looking at AI systems and all the components that come up into that. So I've got a light connection point that we got here, which is once an AI system can access tools, data, identities, and other systems, does attacking it start to look a lot more like traditional offensive security? 

Jorge Calderon: You start. 

Matt Burrough: For me or for JC? 

Elliot Volkman: Whichever one you want to start with. 

Matt Burrough: I can go first and then you can -- 

Jorge Calderon: Sure. 

Matt Burrough: -- totally add on and correct me. You know, I think we see a lot of the attacks start with AI right now. So we'll see things like going against chatbots, that's a new attack surface that the attackers are looking at. If you can pivot through those and convince those systems to give you access to more things, then you tend to move into more of the traditional access. And so for, you know, we've seen a lot of initial access through AI and then pivoting into traditional on-prem attacks for the final TTPs. 

Jorge Calderon: Yeah. A good observation around that is, typically when security products aren't really up to snuff on the way some of the shenanigans is done in agentic ways, we'll start to see detections and alerts later in the attack chain. And that's not something you typically want to see. You want to start getting alerted like right at foothold. But if the foothold is an agent doing something squirrely because XPIA is involved, then you're moving your opportunity for disruption later in the attack chain, which means the attackers had more time to do whatever it is they were trying to do. 

Elliot Volkman: So maybe I can put out a relevant thing that has been in the headlines. So you probably should have -- not you, but something -- 

Jorge Calderon: It wasn't me. 

Elliot Volkman: Something should have caught the 17,000 touch points that a certain set of four agents might have been sniffing around and very -- maybe, possibly. It's a little weird. 

Jorge Calderon: Are we talking the Anthropic here? 

Elliot Volkman: I think it's the OpenAI. 

Jorge Calderon: OpenAI's one? They both have done it at this point. 

Elliot Volkman: Yeah, they're all overlapping. 

Jorge Calderon: Yeah, yeah, yeah. 

Elliot Volkman: But I think there were 17,000 different points of action that they captured across four agents in four days. I'm like, yeah, it's a little slow to have caught it, right? 

Jorge Calderon: Without throwing too much shade, I think -- 

Elliot Volkman: I'm just -- all for technical and educational purposes. 

Jorge Calderon: Yeah, yeah. The agents will do whatever you give them access to do. So like getting out to the web, it just means the sandbox that they had described just wasn't as sandboxy as it needed to be. And so it becomes -- 

Elliot Volkman: That part. 

Jorge Calderon: Yeah, I get -- I could definitely see this happening if a security practitioner wasn't involved and some oversights around the sandbox were missed. 

Elliot Volkman: You want to get in trouble with me? 

Matt Burrough: Yeah, sure. Yeah, I think -- I'm not intimately familiar with the attack, like I haven't, you know, been working on it at all. But you know, from what I've seen, it sounds like very latest cyber models doing offensive things against Defender models that were maybe a couple generations older or less advanced. And so, you know, you start to see the asymmetry there and where it becomes really important that you're keeping your defense AI as strong as the attack AI that exists. 

Elliot Volkman: Yeah, that makes sense. So, changing back gears to the focus, the topic at hand, I think one of the more interesting things that we're seeing is like, with AI, there's always this argument of there's going to be these really unique and novel things that come out into the world, but as you're going through this process, are you feeling like a lot of what you're seeing is renewed concepts that are just kind of changing tactically or across different platforms, or is there a truly unique, new challenges that we're having to, you know, be on the lookout for here? 

Jorge Calderon: We're definitely seeing both. We're seeing kind of like remixes on the traditional attacks. So things that now incorporate AI that probably don't need to or didn't previously incorporate AI. And we're also seeing new services, new app services that are AI powered. There's new ways to attack some of these things, ways that weren't typically in like the OWASP top ten, especially if they weren't LLM enabled in the past. So you weren't previously able to do like a prompt injection against a web app. Now you can if that web app, you know, leverages AI somewhere. 

Matt Burrough: Yeah. And I think we're also seeing like some what old is new again type things where, you know, it's still learning and being built, and so things like social engineering where, you know, you've probably trained your support agents for, you know, a hundred hours not to be socially engineered, maybe the AI hasn't been trained in that way, so. 

Elliot Volkman: First of all, I love the remix positioning, I'm stealing that. That is a way better way to call it. But I've got a kind of like a dumb question, it's a little fuzzy on how we're going to position it, but obviously we attack the models, we attack the harnesses in the systems, but I'm curious how you perceive, like, what if we stop attacking those pieces of technology, but the underlying trust that, like, the guardrails that function around it? Like, what happens when we start abusing and attacking the trust and trying to break it? And it's not just being like prompt injections. And again, very fuzzy, but trust is a pretty core concept through this AI system that I think, like, adds a new level of uniqueness or maybe a remix. 

Jorge Calderon: Yeah. Trust is one of those things, you know, a lot of people say zero trust, but then like they say the words but they don't do the thing. 

Elliot Volkman: Yeah. 

Jorge Calderon: And so the problem there is what you have is a very unintelligent infant that has a really good vocabulary, right? It's essentially social engineering. I socially engineer a human and I get them to click something that is malicious. I socially engineer an agent and I need it to do something for me on my behalf, the attacker, right? And so I really much look at it as the extension, the next area of social engineering is, what's the text going to -- you know, what's it going to take from a textual delivery to make this agent, I don't know, give me an OAuth token or give me the contents of a SharePoint or something like that? It's social engineering under the hood, that's basically what it is. 

Elliot Volkman: Love that positioning. Unfortunately, social engineering has gone from psychological to now truly technological, because now agents perceive it the same way. So it'll be interesting to see how that gets tackled as things progress. I am going to shift this over a little bit over to, you know, obviously you're doing Purple teaming exercises. There are scenarios where the Red team wins in this case, but finding a weakness is obviously only half of that exercise. Once the offensive team proves that something is functioning as you all had hypothesized, how do you figure out whether the security team actually saw it and then respond to it? 

Matt Burrough: Yeah, I think that's, you know, the core piece of Purple teaming is that we're not the kind of Red versus Blue, you know, separate teams. We're literally sitting in the same team, on the same calls, you know, we're buddies, we're working together. And so by taking that approach, there's no, you know, shame or harm in, oh, you didn't detect this. It's, you know, that's the whole point. We're looking for things that don't get detected and making sure that we make sure they are detected in the future. And so just having that Blue teamer ride along with you and be there side by side with you as an offensive person is great, because they can see in real time, hey, did the telemetry come in? Is telemetry delayed? Does it not exist? You know, are we not monitoring the right things? And just be able to improve and file those kinds of bugs of, you know, hey, this isn't getting captured in the logs, I need this information to be able to run down this attack and they don't have it. So getting that right in the moment versus like, you know, Red team, Blue team, the more traditional approach where it's kind of a separate process, Blue team comes in eventually and responds, they don't always have a clear picture exactly what you did. You know, sometimes there's a time delay there and they can't go back and see necessarily what they missed, and so you just get much better clarity as a Purple team. 

Elliot Volkman: That makes sense to me. So, in a typical simulation, it can involve like 30 to 100 maybe sort of discrete attack steps as part of the chain. Which then have to be correlated against alerts, incidents, telemetry, and analyst observations. I'm curious how we're sort of like capturing that information and parsing it and then building maybe defensive measures. Jorge, you want to maybe add any, like [inaudible 00:36:54]. Jorge Calderon: That one's for him. 

Matt Burrough: Yeah, that one's for me. You know, we started doing it manually, and of course it's hard, it takes time. We want to move faster from the speed of AI and all those things. And so one of the things our team has built internally, we're calling BlueAgent, which is a really cool concept of having you know, basically this Blue teamer, but who's a virtual, AI Blue teamer riding along with the team doing the rest of the work with us. And so it's taking in the list of TTPs that we're running as the attacker, and it's also taking in all that telemetry, all the alerts that did fire, and then doing the mapping for us and saying, okay, based on this attack against this account on this machine, I see this alert, that's a one-to-one, we're great, you know, that we have perfect coverage for that. This one, I'm seeing maybe some alerts that are a little bit more vague. They don't call out the, you know, IP it came from or the host name or whatever it was, or it just says it was an attack, it doesn't specifically call out what kind of attack. We should probably improve that detection and make sure it's much more clear for the SOCs so they can make it more actionable. 

Elliot Volkman: I'm curious, how does that change the dynamics? Because obviously, in a past life, it was just people working together as a, you know, a joint operation, but how's it functioning now that you have an agent functioning at your direction? 

Matt Burrough: Yeah, I think it, you know, greatly improves our speed. It's taken our, you know, that kind of process of correlation down from, you know, for some of these ops, for example, from like 20 to 30 hours down to a few hours, which is great. It really lets us focus more on the specific findings that we want to make sure are getting logged properly and not just the, you know, kind of tedious work of going through tons and tons of telemetry. So just being way more efficient in the work that we do. 

Elliot Volkman: Yep. Jorge, anything that you want to add to that? How is it working with a agent colleague? 

Jorge Calderon: It's interesting. Like, I'm a Red teamer by trade, so I really never gave security products, they were just in my way and I just had to get around them. So it is interesting, like the new workflow of like everything goes back to, like, the EDRs and what could they have done better, how can you improve it, and this, like, this whole side of it is new to me, like not ever being in Blue. Yeah. 

Elliot Volkman: Has that maybe changed your perspective on how you're actually like going through the Red team system, since now you're a little bit closer to it? 

Jorge Calderon: We may open -- if I ever did go back into, like, complete Red team ops, it may be like more of a focus on providing feedback to the defenders on here's what that detection missed or here's a way it could be improved. 

Elliot Volkman: That makes sense to me. All right, let's change gears a little bit. So we're going to cover where do security researchers actually trust AI, and I feel like you're doing a good job of painting a picture of where they're partnering and functioning at your direction. But obviously there are challenges, different philosophies that come into mix here. So researchers obviously deliberately do not let LLMs or AI to calculate things like false negative rates, detection coverage, those remain deterministic in most cases, and AI generated correlations remain reviewable by humans, for important reasons. So this begs the question, if you've built AI into the research workflow, but you've also been very deliberate about things you won't let AI do, where is the line? 

Matt Burrough: Yeah. That's really where MCP servers come in. So you can write a custom MCP server that the AI invokes, and that is deterministic code that gets run. And so we can say, hey AI, any time you're going to give us the true positive rate, a false negative rate, go call this MCP server, it'll do the math for you, it'll make sure it's right. So that guarantees those are right. Another place we've seen that a lot is like MITRE attack TTPs, like the attack IDs. We like to correlate all of our TTPs back to MITRE just for tracking purposes. And if you let AI add it, it will hallucinate MITRE TTPs. And so writing a custom MCP server that has, you know, the canonical MITRE attack framework in it and can reference that, you guarantee to get the right thing. 

Elliot Volkman: That's actually really good context. Because, arguably, if you're taking a bunch of information, it should in theory be able to map to ATT&CK or ATLAS or whichever framework we're using for MITRE. So is that going to be like the same answer that we want to articulate for, like, it should be able to calculate detection coverage, but we just can't trust it right now? 

Matt Burrough: Yeah. I mean, that's where you evoke -- 

Elliot Volkman: Deterministic code. Perfect. All right, simple answer. Love it. So let me maybe just tag one on there, which is there's currently so much emphasis right now on giving agents more autonomy. Has your experience made you more comfortable with agent autonomy or more cautious, or just somewhere in the middle?  

Matt Burrough: I mean, I'm wearing -- like I said, my -- 

Elliot Volkman: Is it a can of worms? 

Matt Burrough: -- took my badge off for this one. I do have the, like, professionally paranoid sticker on my badge. So, you know, I think -- Jorge Calderon: I mean, we're security practitioners, so like, even agents, anything we're talking about, our brain goes to, how's this going to be abused? Like, what's the thing they didn't think about? 

Elliot Volkman: Yep. 

Jorge Calderon: I think the big consideration is just the stuff's being rolled out so quickly, it's being baked into so many workflows. And it's so powerful. It's so powerful. The things it can do once it get -- once you get code flow -- sorry, you get logic flow in an agent, you can make it do anything you want just through text. So, if anything, I'm still very reserved as far as like things that I have that I would want to release. I also don't know what some of the inference endpoints, what they collect and what they use to keep on, you know, from a privacy perspective. So those would be like primary concerns of mine. 

Elliot Volkman: All right. Change gears. We've got offensive researchers using AI systems as the target, and also the defensive side increasingly using AI as part of their own workflow. Does that fundamentally change how quickly a Purple team can learn? And I feel like you've already articulated that to an extent, but I'm curious, like, speed to necessity, like how has that truly changed from a time factor? 

Matt Burrough: Yeah, I mean, I think it's, like I said, it's definitely improved our report writing times, our correlation times. You know, AI's also a big area for us to focus on from a security research perspective, and so it is consuming time, and that, you know, that's something we're spending a lot of time thinking about and trying to improve. So both, I guess, is kind of the answer. We are using AI, you know, several ways throughout kind of the Purple teaming process, including, you know, on the correlating, like the Blue side, but then also helping us generate the TTPs and generate the signals on the Red side. So it is helping there as well. That's definitely a kind of slower area of research. It's a part where LLMs generally are not trained to do that, and so that's -- it takes a bit more work to make those work. But yeah. 

Elliot Volkman: Cool. So maybe, hopefully not -- that's probably not a can of worms here, but it will be the audience and the area around us, which is there's a lot of fear and uncertainty about the positioning of what AI and agents are positioned to do in this moment versus what they could do in the future. But I'm curious how you perceive it in this time. Because obviously you're now directing agent and AI to work on your behalf, but do you feel like it is truly trying to take work away from analysts, or is it really helping to investigate more things and iterate faster? So it is another piece of technology, your tool, it's not some magical thing to -- 

Matt Burrough: Absolutely. I mean, shoutout, my team is hiring right now, so clearly it's not taking away our work, right? So if anyone is looking for a job in threat intelligence or Microsoft Security Research, aka.ms/msecrjobs will get you links to our JDs, go apply. So. 

Elliot Volkman: That is the only sales pitch I'm going to let you can do. 

Matt Burrough: Fair. 

Jorge Calderon: He's good at them though. 

Elliot Volkman: Yeah, that was good. We always need more folks on the horde. That's good. Yeah, I appreciate that context, because I think that is just really easy to see all the different technology and vendors as we're walking around Black Hat where, you know, there are upstart, some people might not have been ingrained in our community. And they're just pretending that they can replace an entire SOC with a few agents, which a little iffy. But I don't feel like that is the philosophy that we -- 

Matt Burrough: I mean, for all the agentic work we're doing, each process that we're doing is human in the loop, right? We aren't just trusting AI to go and run and do it end to end. It's always we'll provide some inputs, it'll do some work on our behalf, we'll go check that work, add human intelligence to it, right? Add our expertise. 

Elliot Volkman: Right. 

Matt Burrough: That's what we're paid to do, is to give that oversight and that expertise. 

Elliot Volkman: Yeah, and I'll give a shoutout to your org under TMA, because there's so many pieces of research that we have now published on the npm supply chain attacks. 

Matt Burrough: Yeah. 

Elliot Volkman: And you all, not just building internal agents to help you move faster, but there are also AI workflows that the teams are using. I feel like they were, as in 30 to 60 seconds, detecting any shifts and changes and checking millions of different points of contact to basically just sniff out anything that is happening. So before anyone else in the world really is aware, they're able to basically just immediately sniff something out and we get another article we have to publish within 12 hours [laughs]. 

Matt Burrough: Yeah. 

Elliot Volkman: Cool. All right. So let me shift us towards maybe wrapping this conversation up a little bit. But as companies are moving incredibly quickly to deploy AI agents and AI powered applications, what is one question security teams should be asking about those systems that you don't think they are asking enough right now? 

Jorge Calderon: I think, for me, it'd be like, know the exposure of your agents. Like, can you answer that question about all the ways your agents are taking inputs, where they're getting their instructions from, how exposed though those are to the internet, to the public, and how much do you enable your employees to go and set those things up on their own without too much, I don't know, what's the term, like regulation or -- 

Matt Burrough: Oversight. 

Jorge Calderon: Oversight, yeah. 

Elliot Volkman: Anything you'd like to add? 

Matt Burrough: No, I think that pretty well covers it. Just that and, you know, is this the right model? Is this the right tool for this job? Like, we kind of already talked about, you know, deterministic versus non-deterministic, making sure you're doing the right things in the right places, and trusting where you should and not where you shouldn't. 

Elliot Volkman: Cool. So maybe I'll just add on to that a little bit. But so you're obviously attacking multiple different systems, but I imagine in that mix are AI models and technology using it. I'm curious, in that pursuit of partnership and the proper team operations, is there anything that has shifted your mindset about what an end user might be experience now that you have been fully behind the scenes picking it apart and seeing how these things function? 

Jorge Calderon: We do do evaluations of agents and so we kind of get early access to how some of these security systems work before the customers do. So we do get to provide feedback and, I don't know, if an agent is not performing the way we would or something like, that's something our team is kind of become an expert on that, on the org ad, and they'll come in and get an attacker's kind of perspective on how the agent's performing. So we do directly contribute to, like, the customer experience in that way. Because we know those agents will eventually be in our Azure portal or in a portal somewhere doing what, you know, we've kind of influenced them to do or operate a bit better. 

Elliot Volkman: Matt, any closing thoughts, parting words? 

Matt Burrough: Yeah, I mean, I think it's a learning experience, right? It's a new frontier that we're all getting used to, and so the more hands-on you get, kind of dive into it, you can get a sense of what it's capable of doing, where you feel like it should be used and where it shouldn't, and kind of start to make those choices. 

Elliot Volkman: Perfect. All right. 

Jorge [inaudible 00:49:04], Matt, thank you all so much for joining this episode of the "Microsoft Threat Intelligence Podcast". We will hopefully have you back on in the future as maybe you break some more things and fix some things. Yeah. 

Matt Burrough: Thanks, Elliot.