Black Hat preview: "Vulnerability Research in the Agentic Age."

Dave Bittner: From the earliest days of cybersecurity, vulnerability research has often been viewed as equal parts science and instinct. The best researchers develop a feel for where software is likely to break, combining technical expertise with creativity and persistence, but what if that process could be understood, measured, and even systematized? I'm Dave Bittner, and this is a CyberWire Special Edition. Today we're joined by Yan Shoshitaishvili, Associate Professor at Arizona State University, for a conversation inspired by his Black Hat 2026 Keynote, "Vulnerability Research in the Agentic Age." Yan explores the scientific foundations of vulnerability research and reflects on a career that's uncovered thousands of vulnerabilities across an extraordinary range of technologies, from IoT devices to web browsers, kernels, and boot loaders, using dozens of different tools, techniques, and paradigms. We'll talk about how the field has evolved, what agentic systems may mean for the future of vulnerability discovery, and why understanding the science behind the practice matters for the next generation of security research. Stay with us. [ Music ]

Yan Shoshitaishvili: So my lab has been doing vulnerability research for a very long time. I've been in that space since my graduate studies, and I started my graduate studies in 2010, and basically, ever since then, I've been analyzing different types of systems at very deep levels for vulnerabilities. During this, I created, along with my colleagues, the angr binary analysis framework, a lot of other techniques and approaches to find bugs in software, to understand bugs in software, to remediate bugs in software. We've just been pushing the envelope for a long time, so along comes this new technology, which is LLM inference for bug hunting. In about 2023, the first kind of rumbles of, hey, this is actually pretty interestingly useful come along. It's just accelerated since then. So now, both in our research capacity and in actual security competitions, OODA competitions like, in the style of Pwn2Own, and so on, we are very, very much everywhere using agentic security analysis to augment, sometimes supersede, but mostly augment our existing work. I figured, a lot of experience from my vantage point that I can draw on because, you know, as a faculty researcher, I work with dozens of students and other researchers in the space. There's a lot of lessons learned that we've taken away from a lot of different projects, and it'd be cool to spread those lessons to the community.

Dave Bittner: Well, before we dig into the details of your presentation, it helped me understand how vulnerability research has been considered in the past. I mean, to what degree was it thought of as a technical capability? What degree was it a craft or even an art form?

Yan Shoshitaishvili: It's a great question because it's been considered all of these different things to different extents. When I started, it was 2010, I started as a graduate researcher, but of course, I was into security before that and the CTF scene, and so on. To find bugs in software, when I started, you could rely on these static analyzers with high false positives, unclear usability. You could dig into the very, very early days of dynamic analysis with sending random inputs, hoping for the best, or generating file formats, hoping for the best, hoping to observe security flaws in an application, but really, the majority of bug hunting back then was manual. You would stare at this software, and you would understand it, and that was very much a, kind of, person against software, like almost martial art. In fact, inspired by that sort of feeling of vulnerability research as a martial art in the early days created a whole martial art and security training with my pwn.college platform that really is inspired by this feeling, like, this is an art form. Then gradually, this art form became a science. As new technology came up, I would say, the biggest one that created, in my view, a really -- a very well-defined practice of this is kind of the science, the specific techniques, would be the rise of very standardized fuzzers, like American Fuzzy Lop. There were some before that, but a lot arose right at the same time, but around the time of DARPA's Cyber Grand Challenge Competition, the world of fuzzing kind of exploded, and now you have very well-defined. Hey, if you want to analyze this type of software, well, this is what you do. You set up this fuzzer. You add these types of seeds. You configure this type of sanitization, this type of code coverage. It became a much more step-by-step kind of science, let's say, than a more vague art. There's still space for the art, so there's this merger of both, right? As people built up understanding of the software that they were analyzing, they could really dig in much more cleverly. There's a lot of space for this human expertise and intuition. There was a huge equalization. You could really, without knowing much about reverse engineering, and so on, start finding bugs in real software. We saw a huge increase in the number of vulnerabilities identified and disclosed, and so on, in that time. That era, I would claim, is ending now, where the hacker with a fuzzer is starting to be out-competed by hackers wielding AI agents that are wielding fuzzers. Now in this newly discovered era, we're in the art and invention phase of this, but it's going to resettle into its own very well-regimented, "This is how you analyze software in the modern day," as well.

Dave Bittner: Does the person who has the background in vulnerability research, the person who in the previous world was a gifted artist, do they still have an advantage using the agentic tools?

Yan Shoshitaishvili: Absolutely, especially in this phase. I'm going to say something that can probably be -- can certainly be criticized, and I would probably call people out if they said stuff like this. In March, Anthropic released a sneak peek into Mythos, their super cyber-capable model with a lot of fanfare. This is how many bugs Mythos finds in this target, this is how many bugs Mythos finds in this target and so on. We were already, also, of course -- as was much of the rest of the world, doing agentic-driven vulnerability research. We look at these numbers, as did a lot of other people, and say, well, that -- wait. We're getting similar numbers without Mythos. Now, the difference, of course, is the amount of kind of human expertise, target-specific expertise, really seed -- not seeds in the traditional fuzzing sense of inputs into a program, test cases to drive different behavior, but seeds as in insights, different insights, given to the models and the agentic pipelines that enabled capabilities that seem well beyond the base capabilities and models. I have friends that achieve, quote, "Mythos-like results with open models." We've achieved, in many different projects, quote, "Mythos-like results" with frontier models and so on. The differentiator there is that human insight/human intuition for now. It's unclear if that will remain. I don't see it fully going away anytime really soon. You're seeing a bit of a schism with hackers, actually, and how they approach this. I'm very active in the competitive Capture the Flag community. I ran DEF CON CTF for a couple of years. I've been a core member of Shellphish for way too long now. I think 17 years now. That's terrifying, so I've seen a lot of different trends in the CTF community. When we came out with the angr binary analysis framework, I mentioned, it minorly, compared to LLMs, minorly -- in a minor way -- revolutionized the reverse engineering Capture the Flag category. When decompilers came out and got good, they revolutionized the, you know, also, reverse engineering and the exploitation category, the binary exploitation category of CTFs, as well as revolutionizing reverse engineering in the real world. In CTF, what you saw -- what I saw, because I have been in CTF for that long -- with the rise of decompilers is there were hackers that refused to adopt them, that really liked looking at assembly instructions instead of pseudocode. Some of the best of them could keep up for a while, but if you didn't adopt the latest tools, you were eventually kind of forced to, basically, announce your retirement. We're seeing that right now in the Capture the Flag community. All of these hackers are retiring because, to them, what they really loved about Capture the Flag, about that sort of applied high-stakes security, is being kind of eaten away by the agents doing the nitty-gritty that they really loved to do. We're also seeing top CTF teams with the remaining players adapt to the modern paradigm, because if you look at the scoreboard of the top CTFs of, for example, DEF CON CTF qualifiers, there's a quick competition with bespoke software, so it's not fully representative of the real world, but there's a window to the real world in it. You see the top teams remain the top teams because they really invest in understanding how to have the human value add, how to properly harness the modern technology with agents and security analysis by agents, to use their human expertise to improve over the purely agent expertise. What I see, although less of this happens in the open outside of the CTF community, I see the same thing outside of the CTF community. I see the same thing in our research labs. I see the same thing in our spinoff companies. I see the same space for human ingenuity and insights and expertise to still be very, very, very valuable.

Dave Bittner: What do you think this means for the future of vulnerability research? I mean, are we going to see the same type of person attracted to this but using tools in a different way, or might we see a different set of people attracted to this particular area?

Yan Shoshitaishvili: I think it won't be like a disjoint set of people, but it won't be the same set. We're going to see different people being attracted to the area to different extents in different ways. I was talking with a former student of mine who's now faculty elsewhere. He mentioned a mentee of his that had found a zero-day vulnerability in a complex real-world system without having expertise in the types of pieces of the core security micro-scales that we would expect traditionally. This is awesome because it broadens the field in a similar way that decompilers broaden the field, but of course, a much more fundamental way, in my opinion, than the decompilers. I don't think we've seen a shift to this fundamental, but it is not completely without precedence. We're going to see a lot more people approaching the field, getting interested in cybersecurity, but we're going to see them be slightly more aloof of the really, really deep details, again, in the same way that people that reverse engineer purely through a decompiler might be relatively ignorant of the subtleties at the assembly level. That's going to be a very interesting transition. I think the amount of people that get into cybersecurity is going to grow because this stuff is really fun, whether you're doing it through -- with the help of an agent or not. Realistically, again, we've always been doing it with the help of tools for decades. This is, in that sense, kind of another tool. I think it'll drive a lot of the people that really love those specific micro skills. It'll drive them out. It has already driven them out. They see a lot of old timers in cybersecurity, at least leave Capture the Flag because they feel like what they love about it is no longer relevant. I think for some of them, it's a good excuse to put down the cape and go to a less stressful life, but will those people leave security in general? I have my doubts there. I think a lot of them are still leading security companies, etc., etc., exploring the new capabilities. I think in the end, the current old guard will probably stay because the current old guard at the high level doesn't look at assembly anymore, anyways. A lot of new people are going to flood in because they can now. It's much more approachable. I mentioned I run a cybersecurity education platform. It's open to the world for free, and we see a lot more interest in the higher levels because people -- it used to be that people might get stuck working through a problem for a month. Now they don't, which is unfortunate in terms of developing those micro skills, but in terms of getting them through the high-level concept seems to work interestingly well. That population is growing. Despite a lot of fears that AI will eat security, this hasn't happened yet. It could, potentially, especially as the models get better. It's hard to predict what happens. There's my personal way that I think of the definition of the AI singularity is a point past which it becomes impossible to predict anything. You just kind of have to go with reasonable assumptions that you can make. It'll shape over time. I think we won't lose the old guard that was interested in assembly, but we might lose the new guard that is interested in assembly. They might go more into architecture design, things that are a little harder to agentify than security if they're really in it for the specific low-level concepts, like the very deep foundations of computing, which is why I am into security. In the introduction of my dissertation, I wrote about how the magic of computing to me happens at the assembly level. Somewhere there, there's, like, a magic that ignites and these very simple instructions come together to create very complex systems that have fascinating security implications in that the security implications are like corner cases and failure modes of this emergent, complex behavior. If people are no longer working at that level, they might gradually move into other fields. I might have moved into architecture or something instead of cybersecurity if I was coming up nowadays.

Dave Bittner: At the risk of putting words in your mouth, it seems as though part of what you're saying, you're kind of, expressing a cautionary tale to the old timers: Despite any nostalgia we may have for the old ways, you know, let's not be outright bitter and resentful for the new tools that are coming along. I mean, is that accurate?

Yan Shoshitaishvili: Absolutely. The security professionals, and this, again, is much more public in the CTF world because they go on Twitter and announce their retirements. I see very similar things in the professional world, or at least examples of them. The new tech just enables a reach and a capability that simply didn't exist before. There is an incredible amount, as I'll talk in my talk at Black Hat, an incredible amount of little friction areas that agents completely eliminate, allowing better use of the tools we already have. It's similar to if you hired a thousand security engineers, and let them lose with tooling on a problem, right? These are incredible to see when you actually are observing them in action, and people focus on the obsolescence of specific individual micro skills. Of course, there are various concerns about the externalities of the technology, but from a purely cybersecurity perspective, if we can ignore the externalities, which, of course, is tricky from purely cybersecurity perspective, we are living in a renaissance that I haven't seen, really, maybe ever, but definitely not since, like, the rise of capable decompilers and so on. [ Music ] Our thanks to Yan Shoshitaishvili, associate professor at Arizona State University, for joining us and sharing insights from his Black Hat 2026 keynote, "Vulnerability Research in the Agentic Age." If you enjoyed this conversation, be sure to subscribe to the CyberWire so you never miss out on our special editions, daily news, and in-depth interviews with the people shaping the cybersecurity landscape. I'm Dave Bittner. Thanks for listening. [ Music ]