Ready or Not, Here A.I. Come!

Nic Filingham: Hello and welcome to Security Unlocked! A new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft Security Engineering and Operations Teams. I'm Nic Filingham.

Natalia Godyla: And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security. Deep dive into the newest threat intel, research, and data science.

Nic Filingham: And profile some of the fascinating people working on artificial intelligence in Microsoft Security.

Natalia Godyla: And now, let's unlock the pod.

Nic Filingham: Hello, Natalia! Hello, listeners! Welcome to episode 26 of Security Unlocked. Natalia, how are you?

Natalia Godyla: Thank you, Nic. And welcome to all our listeners for another episode of Security Unlocked. Today, we are chatting about gamifying machine learning, super cool, and we are joined by Christian Seifert and Joshua Neil who will share their research on building CyberBattleSim, which investigates how autonomous agents operate in a simulated enterprise environment by using high-level obstruction of computer networks and cyber-security concepts. I sounded very legit, but I did just read that directly from the blog.

Nic Filingham: I was very impressed.

Natalia Godyla: (laughs)

Nic Filingham: If you had not said that you read that from the blog, I would've been like, "Wow". I would to like to subscribe to a newsletter.

Natalia Godyla: (laughs)

Nic Filingham: But this is a great conversation with, with Christian and Joshua. We talked about what is reinforcement learning. Sort of as a concept and how does that gonna apply to security. Josh and Christian also walked us through sort of why this project was created and it's really to try and get ahead of a future where, you know, malicious actors have access to some level of automated, autonomous tooling. Uh, and so, this is a new project to sort of see what a future might look like when there all these autonomous agents out there doing bad stuff in the cyber world.

Natalia Godyla: And there are predecessors to this work, at least in other domains. So, they used a toolkit, a Python-based Open AI Gym interface to build this research project but there have been other applications in the past. OpenAI is, uh, well-known for a hide-and-seek. There is a video on YouTube that shows how the AI learned over time different ways to obstruct the agent and the simulated environment. Things like, blocking them off using some pieces of the wall or jumping over the wall.

Nic Filingham: The only thing we should point out is that this CyberBattleSim is an open source project. It's up on GitHub and attained very much want researchers, and really anyone who's interested in this space to go and download it, go and run it, play around with it, and help make it better. And if you have feedback, let us know. There is contact information, uh, through the GitHub page but you can also contact us at Security Unlocked at Microsoft dot com and we can make sure you, uh, get in contact with the team. And with that, on with the pod?

Natalia Godyla: On with the pod!

Nic Filingham: Welcome to Security Unlocked, new guest, Christian Seifert. Thanks for joining us and welcome returning guest, Josh Neil, back to the podcast. Both of you, welcome. Thanks for being on Security Unlocked.

Christian Seifert: Thanks for having us!

Joshua Neil: And thanks, Nic.

Nic Filingham: Christian, I think as a, as a new guest on the podcast, could we get a little introduction for our listeners? Tell us about, uh, what you do at Microsoft. Tell us about what a day to day look like for you.

Christian Seifert: Sure, so I'm a, uh, research lead on the Security and Compliance team. So our overall research team supports a broad range of enterprising consumer products and services in the security space. My team in particular is focused on protecting users from a social engineering attack. So, uh, think of, like, fishing mails for instance. So we're supporting Microsoft Defender for Office and, um, Microsoft Edge browser.

Nic Filingham: Got it, and Josh, folks are obviously familiar with you from previous episodes but a, a quick re-intro would be great.

Joshua Neil: Thanks. I currently lead the Data Science team supporting Microsoft threat experts, which is our managed hunting service, as well as helping general res... cyber security research for the team.

Nic Filingham: Fantastic, uh, again, thank you both for your time. So, today in the podcast, we're gonna talk about a blog post that came out earlier in this month, on April 8, called Gamifying Machine Learning for Stronger Security in AI Models, where you talk about a new project that has sort of just gone live called CyberBattleSim. First off, congratulations on maybe the coolest name? For, uh, sort of a security research project? So, like, I think, you know, just hats off there. I don't who came up with the name but, but great job on that. Second of all, you know, Christian if, if I could start with you, could you give us a sort of an introduction or an overview what is CyberBattleSim and what is discussed in this blog post?

Christian Seifert: As I... before talking about the, the simulator, uh, the... let me, let me kind of take a step back and first talk about what we tried to accomplish here and, and why. So, if you think about the security space and, and machine learning in particular, a large portion of machine-learning systems utilized supervised, uh, classifiers. And here, essentially, what we have is, is kinda a labeled data set. So, uh, for example, a set of mails that we label as fish and good. And then, we extract, uh, threat-relevant features. Think of, like, maybe particular words in the body, or header values we believe that are well-suited to differentiate bad mails from good mails. And then our classifiers able to generalize and able to classify new mails that come in.

Christian Seifert: There's a few, uh, aspects to consider here. So, first of all, the classifier generalizes based on the data that we present to it. So, it's not able to identify completely unknown mails.

Christian Seifert: Second, is that usually a supervised classification approach is, is biased because we are programming, essentially, the, the classifier and what it, uh, should do. And we're utilizing domain expertise, red teaming to kind of figure out what our threat-relevant features, and so there's bias in that.

Christian Seifert: And third, a classifier of who has needs to have the data in order to make an appropriate classification. So, if I have classifier that classifies fish mail based on the, the content of the mail but there is the threat-relevant features are in the header, then that classifier needs to have those values as well in order to make that classification. And so, my point is these classifiers are not well-suited to uncover the unknown unknowns. Anything that it has not seen, kinda new type of attack, it is really blind to it. It generalizes over data that, that we present to it.

Christian Seifert: And so, what we try to do is to build a system that is able to uncover unknown attacks with the ultimate goal then to, of course, develop autonomous defensive component to defend against those attacks. So, that gives it a little bit of context on why we're pursuing this effort. And this was inspired by reinforcement learning research and the broader research community, mostly that is currently applied kinda in the gaming context.

Christian Seifert: So OpenAI actually came out with a neat video a couple of years ago called Hide and Seek. Uh, that video is available on YouTube. I certainly encourage listeners to check it out, but basically it was a game of laser tag where you had a kinda, uh, a red team and a blue team, uh, play the game of laser tag against each other. And at first they, of course, randomly kind of shoot in the air and run around and there is really no order to the chaos. But eventually, that system learned that, “Hey, if a red team member shoots a blue team member, there's a reward.” and the blue team member also learned while running away from the red team member is, is probably a good thing to do.

Christian Seifert: And so, OpenAI kinda, uh, established the system and had the blue team and the red team play against each other, and eventually what that led to is really neat strategies that you and I probably wouldn't have come up with. 'Cause what the AI system does, it explores the entire possible actions base and as result comes up with some unexpected strategies. So for instance, uh, there was a blue team member that kinda hid in a room and then a red team guy figured, “Hey, if I jump on a block then I can surf in that environment and get into the room and shoot the blue team member”. So that was a little bit an inspiration because we wanted to also uncover these unknown

Christian Seifert: Unknownst in the security context.

Nic Filingham: Got it. That's great context. Thank you Christian. I think I have seen that video, is that the one where one of the many unexpected outcomes was the, like, one of the, the, blue or red team players, like, managed to sort of, like, pick up walls and used them as shields and then create ramps to get into, like, hidden parts of the map? Uh, uh, am I thinking about the right video?

Christian Seifert: Yes, that's the right video.

Nic Filingham: Got it. So the whole idea was that that was an experiment in, in understanding how finding the unknown unknowns, using this game, sort of, this lazar tag, sort of, gaming space. Is, is that accurate?

Christian Seifert: That's right, and so, they utilized reinforcement learning in order to train those agent. Another example is, uh, DeepMind's AlphaGo Zero, playing the game of Go, and, and here, again, kind of, two players, two AI systems that play against each other, and, over time, really develop new strategies on how to play the game of Go that, you know, humans players have, have not come up with.

Christian Seifert: And it, eventually, lead to a system that achieved superhuman performance and able to beat the champion, Lisa Dole, and I think that was back in 2017. So, really inspiring work, both by OpenAI and DeepMind.

Nic Filingham: Got it. I wonder, Josh, is there anything you'd like to- before we, sort of, jump into the content of the blog and, and CyberBattleSim, is there anything you'd like to add from your perspective to, to the context that Christian set us up on?

Joshua Neil: Yeah. Thanks, Nic. I, I mean, I think we were really excited about this because... I think we all think this is a natural evolution of, of our adversaries, so, so, currently, our adversaries, the more sophisticated ones, are primarily using humans to attack our enterprises and, that means they're slow and they can make mistakes and they don't learn from the large amount of data that's there in terms of how to do attacks better, because they're humans.

Joshua Neil: But I think it's natural, and we just see this, uh, everywhere and, all of technology is that people are bringing in, you know, methods to learn from the data and make decisions automatically, and it's- so it's a natural evolution to say that attackers will be writing code to create autonomous attack capabilities that learn while they're in the enterprise, that piece of software that's launched against the enterprise as an attack, will observe its environment and make decisions on the fly, automatically, from code.

Joshua Neil: As a result, that's a frightening proposition because, I think the speed at which these attacks will proceed will be a lot, you know, a lot more quick, but also, being able to use the data to learn effective techniques that get around defenses, you know, we just see data science and machine learning and artificial intelligence doing this all over the place and it's very effective that the ability to consume a large amount of data and make decisions on it, that's what machine learning is all about. And so, we at Microsoft are interested in exploring this ourselves because we feel like the threat is coming and, well, let's get ahead of it, right? Let's go experiment with automated learning methods for attacks and, and obviously, in the end, for defense that, by implementing attack methods that learn, we then can implement defensive methods that will, that will preempt what the real adversaries are doing, eventually, against our customers.

Joshua Neil: So, I think that's, sort of, a philosophical thing. And then, uh, I love the OpenAI Hide-and-Seek example because, you know, the analogy is; Imagine that instead of, they're in a room with, um, walls and, and stuff, they're on a computer network, and the computer network has machines, it has applications, it has email accounts, it has users, it's got a cloud applications, but, in the end, you know, an attacker is moving through an environment, getting blocked in various ways by defenses, learning about those blockings and detections and things and finding gaps that they can move through in, in very similar ways. So, I just, sort of, drawing that analogy back, Hide-and-Seek, it is what we're trying to do in cyber defense, you know, is, is Hide-and-Seek. And so the, I think the analogy is very strong.

Nic Filingham: Josh, I just wanna quickly clarify on something that, that you said there. So, it sounds like what you're saying is that, while, sort of, automated AI-based attacking, attackers or attacking agents maybe aren't quite prevalent yet, they're, they're coming, and so, a big part of this work is about prepping for that and getting ahead of it. Is, is, is that correct?

Joshua Neil: That's correct. I, I'm not aware of sophisticated attack machinery that's being launched against our enter- our customers yet. I haven't seen it, maybe others have. I think it's a natural thing, it's coming, and we better be ready.

Christian Seifert: I mean, we , we see some of it already, uh, in terms of adversarial machine learning, where, uh, our machine learning systems are getting attacked, where, maybe the input is manipulated in a way that leads to a misclassification. Most of that is, is currently more, being explored in the research community.

Natalia Godyla: How did you apply reinforcement learning? How did you build BattleSim? In the blog you described mapping, some of the core concepts of reinforcement learning to CyberBattleSIm, such as the environment, that action space, the observation space and the reward. Can you talk us through how you translated that to security?

Christian Seifert: Yeah. So, so first let, let me talk about reinforcement learning to make sure, uh, listeners understand, kinda, how that works. So, as I mentioned, uh, earlier in the supervised case, we feed a label data set to a learner, uh, and then it able to generalize, and we reinforcement learning works very differently where, you have an agent that sits within an environment, and the agent is, essentially, able to generate the data itself by exploring that environment.

Christian Seifert: So, think of an agent in a computer network, that agent could, first of all, scan the network to, maybe, uncover notes and then they're, maybe, uh, actions around interacting with the notes that it uncovers. And based on those interactions, the agent will, uh, receive a reward. That reward actually may be delayed by, like, there could be many, many steps that the agent has to take before the reward, uh, manifests itself. And so, that's, kinda, how the agent learns, it's, e- able to interact in that environment and then able to receive a reward. And so that's, kinda, what, uh, stands, uh, within the core of the, the CyberBAttleSim, because William Bloom, who is the, the brains behind the simulation, has created an environment that is compatible with, uh, common, uh, reinforcement learning tool sets, namely, the OpenAI Gym, that allows you to train agents in that environment.

Christian Seifert: And so, the CyberBattleSim represents a simple computer network. So, think of a set of computer nodes, uh, the, the nodes represent a computer, um... Windows, Mac OS, sequel server, and then every node exposes a set of vulnerabilities that the agent could potentially exploit. And so, then, as, kind of, the agent is dropped into that environment, the agent needs to, first, uncover those nodes, so there's a set of actions that allows to explore the state space. Overall, the environment has a, a limited observability, as the agent gets dropped into the environment, you're not necessarily, uh, giving that agent the entire network topology, uh, the agent first needs to uncover that by exploring the network, exploiting nodes, from those nodes, further explore the network and, essentially, laterally move across the network to achieve a goal that we give it to receive that final reward, that allows the agent to learn.

Natalia Godyla: And, if I understand correctly, many of the variables were predetermined, such as, the network topology and the vulnerabilities, and, in addition, you tested different environments with different set variables, so how did you determine the different environments that you would test and, within that particular environment, what factors were predetermined, and what those predetermined factors would be.

Christian Seifert: So we, we determined that based on the domaine expertise that exists

Christian Seifert: ... is within the team, so we have, uh, security researchers that are on a Red Team that kind of do that on a day-to-day basis to penetration tests environments. And so, those folks provided input on how to structure that environment, what nodes should be represented, what vulnerabilities should be exposed, what actions the agent is able to take in- in terms of interacting and exploring that, uh, network. So our Red Team experts provided that information.

Nic Filingham: I wonder, Christian, if you could confirm for me. So there are elements here in CyberBattleSim that are fixed and predetermined. What elements are not? And so, I guess my question here is if I am someone interfacing with the CyberBattleSim, what changes every time? How would you sorta define the game component in terms of what am I gonna have to try and do differently every time?

Christian Seifert: So the- the CyberBattleSim is this parametrized, where you can start it up in a way that the network essentially stays constant over time. So you're able to train an agent. And so, the network size is- is something that is dynamic, that you can, uh, specify upon startup. And then also kinda the node composition, as well as ... So whether ... how many Windows 10 machines you have versus [inaudible 00:19:15] servers, as well as the type of vulnerabilities that are associated with each of those nodes.

Nic Filingham: Got it. So every time you- you establish the simulation, it creates those parameters and sort of locks them for the duration of the simulation. But you don't know ... The agent doesn't know in advance what they will d- they will be. The agent has to go through those processes of discovery and reinforcement learning.

Christian Seifert: Absolutely. And- and one- one tricky part within reinforcement learning is- is generalizability, right? When you train an agent on Network A, it may be able to learn how to outperform a Red Team member. But if you then change the network topology, the agent may completely flail and not able to perform very well at all and needs to kind of re- retrain again. And that- that's a common problem within the- the re- reinforcement learning research community.

Natalia Godyla: In the blog you also noted a few opportunities for improvement, such as building a more realistic model of the simulation. The simplistic model served its purpose, but as you're opening the project to the broader community, it seems l- that you're endeavoring to partner with the other researchers to create a more realistic environment. Have you given some early thought as to how to potentially make the simulation more real over time?

Christian Seifert: Absolutely. There is a long list of- of things that we, uh, need to think about. I mean, uh, network size is- is one component. Being able to simulate a- a regular user in that network environment, dynamic aspects of the network environment, where a node essentially is added to the network and then disappears from the network. Uh, all those components are currently not captured in the simulation as it stands today. And the regular user component is an important one because what you can imagine is if we have an attacker that is able to exploit the network and then you have a defender agent within that network as well, if there is no user component, you can very easily secure that network by essentially turning off all the nodes.

Christian Seifert: So in- a defender agent needs to also optimize, uh, to keep the productivity of the users that are existing on the network high, which is currently not- not incorporated in- in the simulation.

Nic Filingham: Oh, that's w- that's amazing. So there could be, you know, sort of a future iteration, sort of a n- network or environment productivity, like, score or- or even a dial, and you have to sort of keep it above a particular threshold while you are also thwarting the advances of the- of the agent.

Christian Seifert: Absolutely. And I mean, that is, I think, a common trade off in the security space, right? There are certain security m-, uh, measures that- that make a network much more secure. Think of like two-factor authentication. But it does u- add some user friction, right? And so, today we're- we're walking that balance, but I'm hoping that there may be new strategies, not just on the attacker's side, but also on the defender's side, that we can uncover that is able to provide higher level of security while keeping productivity high.

Nic Filingham: I think you- you- you have covered this, but I- I'd like to ask it again, just to sort of be crystal clear for our audience. So who is the CyberBattleSim for? Is it for Red Teams? Is it for Blue Teams? Is it for students that are, you know, learning about this space? Could you walk us through some of the types of, you know, people and- and roles that are gonna use CyberBattleSim?

Christian Seifert: I mean, I think that the CyberBattleSim today is- is quite simplistic. It is a simulated environment. It is not ... It'-s it's modeled after a real world network, but it is far from being a real world network. So it's, uh, simplistic. It's simulated, which gives us some advantages in terms of, uh, scalability and that learning environment. And so at this point in time, I would say, uh the simulation is really geared towards, uh, the research community. There's a lot of research being done in reinforcement learning. A lot of research is focused on games. Because if you think about a game, that is just another simulated environment. And what we're intending to do here with- with some of the open source releases is really put the spotlight on the security problem. And we're hoping that the- the reinforcement learning researchers and the research community at large will pay more attention to this problem in the security domain.

Nic Filingham: It's currently sort of more targeted, as you say, as- as researchers, as sort of a research tool. For it to be something that Red Teams and Blue Teams might want to look at adopting, is that somewhere on a road map. For example, if- if you had the ability to move it out of the simulation and into sort of a- a- a VM space or virtual space or perhaps add the ability for users to recreate their own network topology, is that somewhere on your- your wishlist?

Christian Seifert: Absolutely. I think there's certainly the goal to eventually have these, uh, autonomous defensive agent deployed in real world environments. And so in order to get to that, simulation needs to become more and more realistic in order to achieve that.

Joshua Neil: There's a lot of work to be done there. 'Cause reinforcement learning on graphs, big networks, i- is computationally e- expensive. And just a lot of raw research, mathematics and computing that needs to be done to get to that real- real world setting. And security research. And in incorporating the knowledge of these constraints and goals and rewards and things that ... T- that takes a lot of domain research and getting- getting the- the security situation realistic. So it's hard.

Christian Seifert: In the simulation today, it provides the environment and ability for us to train a Red Team agent. So an agent that attacks the environment. Today, the defender is very simplistic, modeled probabilistically around cleaning up machines that have been exploited. So as kinda the next point on the wishlist is really getting to a point where we have the Red Team agent play against a Blue Team agent and kinda play back and forth and see kinda how that influences the dynamic of the game.

Natalia Godyla: So Christian, you noted one of the advantages of the abstraction was that it wasn't directly applicable to the real world. And because it wasn't approved as a safeguard against nefarious actors who might use CyberBattleSim for the wrong reason. As you're thinking about the future of the project, how do you plan to mitigate this challenge as you drive towards more realism in the simulation?

Christian Seifert: That is certainly a- a- a risk of this sort of research. I think we are still at the early stages, so I think that risk is- is really nonexistent as it stands right now. But I think it can become a risk as the simulation becomes more sophisticated and realistic. Now, we at Microsoft have the responsible AI effort that is being led at the corporate level that looks at, you know, safety, reliability, transparency, accountability, e- et cetera, as kind of principles that we need to incorporate into our AI systems. And we, early on, engaged the proper committees to help us shape the- the solution in a responsible fashion. And so at this point in time, there weren't really any concerns, but, uh, as the simulation evolves and becomes more realistic, I very much expect that we,

Christian Seifert: ... be, uh, need to employ particular safeguards to prevent abuse.

Nic Filingham: And so without giving away the battle plan here, wh- what are some other avenues that are being, uh, explored here as part of this trying to get ahead of this eventual point in the future, where there are automated agents out there in the wild?

Joshua Neil: This is the- the core effort that we're making, and it's hard enough. I'll also say I think it's important for security folks like us, especially Microsoft, to try hard things and to try to break new ground and innovation to protect our customers and really the world. And if we only focus on short-term product enhancements, the adversaries will continue to take advantage of our customers' enterprises, and we really do need to be taking these kind of risks. May not work. It's too ... It's really, really hard. And t- and doing and in- in purposefully endeavoring to- to- to tackle really hard problems is- is necessary to get to the next level of innovation that we have to get to.

Christian Seifert: And let me add to that. Like, we have a lot of capabilities and expertise at Microsoft. But in the security space, there are many, many challenges. And so I don't think we can do it alone. Um, and so we also need to kinda put a spotlight on the problem and encourage the broader community to help solve these problems with us. And so there's a variety of efforts that we have pursued over the last, uh, couple of years to do exactly that. So, about two years ago we published a [inaudible 00:28:52] data science competition, where we provided a dataset to the broader community, with a problem around, uh, malware classification and machine risk identification and basically asked the community, "Hey, solve this problem." And there was, you know, prize money associated with it. But I really liked that approach because we have ... Again, we have a lot of d- expertise on the team, but we're also a little bit biased, right, in- in terms of kinda the type of people that we have, uh, and the expertise that we have.

Christian Seifert: If you present a problem to the broader research community, you'll get a very different approaches on how people solve the problems. Most likely from com- kind of domains that are not security-related. Other example is an RFP. So we funded, uh, several research projects last year. I think it was, uh, $450,000 worth of research projects where, again, we kind of laid out, "Here are some problems that are of interest that we wanna put the spotlight on, and then support the- the research community p- to pursue research in that area."

Nic Filingham: So what kind of ... You know, you talk about it being, uh, an area that we all sort of collectively have to contribute to and sort of get b- behind. Folks listening to the podcast right now, going and reading the blog. Would you like everyone to go and- and- and spin up CyberBattleSim and- and give it a shot, and then once they have ... Tell us about the- the types of work or feedback you'd like to see. So it's up on GitHub. What kind of contributions or- or feedback here are you looking for from- from the community?

Christian Seifert: I mean, I'd really love to have, uh, reinforcement learning researchers that have done research in this space work with the CyberBattleSim. Kinda going back to the problem that I mentioned earlier, where how can we build agents that are generalizable in a way that they're able to operate on different network topology, different network configuration, I think is an- an- an exciting area, uh, that I'd love to see, uh, the research community tackle. Second portion is- is really enhancing the simulation. I mentioned a whole slew of features that I think would be beneficial to make it more realistic, and then also kinda tackle the problem of- of negatively impacting potential productivities of- of users that operate on that network. So enhancing the- the simulation itself is another aspect.

Nic Filingham: Josh, anything you wanted to add to that?

Joshua Neil: Yeah, I mean, I- I think those were the- the major audiences we're hoping for feedback from. But a- al- also like Christian said, if a psychologist comes and looks at this and has an idea, send us an email or something. You know, that multidisciplinary advantage we get from putting this out in the open means we're anticipating surprises. And we want those. We want that diversity of thought and approach. A physicist, "You know, this looks like a black hole and here's the m- ..." Who knows? You know, but that's- that's the kind of-

Nic Filingham: Everything's a black hole to a physicist-

Joshua Neil: (laughs) Yeah.

Nic Filingham: ... so that's, uh ...

Joshua Neil: So, you know, I think that diversity of thinking is what we really solicit. Just take a look, yeah. Anybody listening. Download it. Play with it. Send us an email. We're doing this so that we get your- your ideas and thinking, for us and for the whole community. Because I think we- we also believe that good security, uh, next generation security is developed by everybody, not just Microsoft. And that there is a- there is a good reason to uplift all of humanity's capability to protect themselves, for Microsoft but for everybody, you know?

Natalia Godyla: So Christian, what are the baseline results? How long does it take an agent to get to the desired outcome?

Christian Seifert: So the s- simulation is designed in a way that also allows humans to play the game. So we had one of our Red Teamers to actually play the game and it took that person about 50 operations to compromise the entire network. Now when we take a- a random agent that kinda uninformed takes random actions on the network, it takes about 500 steps. So that's kind of the- the lower baseline for an agent. And then we trained, uh, a Deep Q, uh, reinforcement learning agent, and it was able to accomplish, uh, the human baseline after about 50, uh, training iterations. Again, network is quite simple. I wouldn't expect that to hold, uh, as kinda the- the simulation scales and becomes more complex, but that was, uh, certainly an encouraging first result.

Joshua Neil: And I think the- the significant thing there is, even if the computer is- takes more steps than the human, well, we can make computers run fast, right? We can do millions of iterations way faster than a- than a human and they're cheaper than humans, et cetera. It's automation.

Nic Filingham: Is there a point at which the automated agent gets too good, or- or is there sort of a ... What would actually be the definition of almost a failure in this experiment, to some degree?

Joshua Neil: I think one- one is to- to sort of interpret your question as it could be overfed. That is, if it's too good, it's too specific and not generalized. And as soon as you throw some different set of constraints or network at it, it fails. So I think that's a- that's a real metric of the performances. Okay, it- it learned on this situation, but how well does it do on the next one?

Nic Filingham: Is there anything else, uh, either of you would like to add before we wrap up here? I feel like I've covered a lot of ground. I'm gonna go download CyberBattleSim and- and try and work out how to execute it. But a- anything you'd like to add, Christian?

Christian Seifert: No, not from me. It was, uh, great talking to you.

Natalia Godyla: Well, thank you Josh and Christian, for joining us on the show today. It was a pleasure.

Christian Seifert: Oh, thanks so much.

Joshua Neil: Yeah, thanks so much. Lots of fun.

Natalia Godyla: Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode.

Nic Filingham: And don't forget to tweet us at MSFTSecurity, or email us at securityunlocked@microsoft.com, with topics you'd like to hear on a future episode. Until then, stay safe.

Natalia Godyla: Stay secure.