CyberWire Daily at 10: The vulnerabilities, zero‑days, and hardware flaws over the last decade.

Dave Bittner: Hi, everybody. I'm Dave Bittner. Over the past decade, we've watched the cybersecurity landscape transform at an astonishing pace. Attackers found new ways in, defenders adapted, and vulnerabilities from software bugs to hardware flaws became defining moments that reshaped how we think about security. In this special edition of our CyberWire Daily 10th Anniversary Series, Maria Varmazis joins me for a look back at 10 years of vulnerabilities, zero-days, and the lessons they've taught us. From watershed events like WannaCry and Log4Shell, to the growing impact of hardware security and artificial intelligence, we'll explore the flaws that changed the industry and what they tell us about the road ahead. Stay with us. [ Music ]

Maria Varmazis: It is my singular joy and pleasure to welcome, once again, the host of the "CyberWire Daily," Dave Bittner. Thank you so much for joining me today.

Dave Bittner: It's always my pleasure, happy to be back with you here today.

Maria Varmazis: Oh, I'm so glad we get to talk, yet again, in this year of the 10-year anniversary of the CyberWire Daily. We're getting to, I think, the meatiest of the meat-and-potatoes conversation. I think people have been sort of wondering when we would get to this one. Well, it's here, everybody. We're talking about --

Dave Bittner: Hold on, buckle up, buttercup.

Maria Varmazis: Buckle up, buckaroos. We're doing it: vulnerabilities. Truly, the world that so many of us live in, we breathe this, the vulns of the last 10 years, whew. All right. To set the stage first and foremost, neither Dave nor I are security researchers by trade, so do we have every single vuln of the last 10 years and then some memorized? No, we don't. [Dave laughs] However, Dave, I know that you have a good sense of, sort of, the overarching narrative arc of a lot of the biggies, given that you've looked at these and been covering these the last 10 years. We're going to stick to, sort of, the big stories of the last 10 years of vulns. Yeah, I guess let's start with maybe a scene setting first. Where were we 10 years ago when you think about the state of vulnerabilities then?

Dave Bittner: I think, 10 years ago, I think most people thought of vulnerabilities as being an IT problem rather than a company-wide problem. I don't think companies were thinking of vulnerabilities as being business risk the way that they do today. Some vulnerability would be found, and it was up to the IT team to figure out how quickly it needed to be taken care of, how serious it was, and what kind of schedule we could patch that relative to our business needs. It was probably a line item in the quarterly presentation that the IT folks made to their bosses or the board or whatever and said, "Oh, we patched this many bugs this quarter and it's great," but I don't know that security was top of mind because we hadn't really seen any of the really bad things that were coming on the horizon.

Maria Varmazis: Yeah, I would imagine adding to that, there was also an element -- and this is me editorializing a little bit -- but when IT would go to someone up the executive chain and go, so this specific situation, this specific vuln that we've got to patch, it's going to result in some downtime. I imagine back then they would get a lot more pushback than they might now, and of course, I imagine that may depend a lot. I imagine making the business case for why do you have to take this system down to just do a patch? Why is that so important? Can't this wait?

Dave Bittner: Yeah, I think the IT folks were probably considered to be lower on the chain in the pecking order, however you want to describe it, within the business, because again, we hadn't really seen this possibility that the cyber issues could kill your business, so the stakes weren't as high.

Maria Varmazis: Yeah, and I'm thinking, also, about timelines for vulnerability exploitation. I mean, not a secret, in the last 10 years, those timelines have accelerated dramatically. Do you have any recollection of what we were typically looking at 10 years ago?

Dave Bittner: I think it was much -- well, let's see, again, not a practitioner, but my sense is it was probably, more often than not, we'll get to that when we get to it. In other words, we'll get to that when it will have the least impact on the business. I think that meant a lot of things got put off, but there probably weren't serious consequences because of that. That was a reasonable plan back then.

Maria Varmazis: Yeah, so inevitable question here, what changed? Something changed. [Laughter] Can we point to a singular vuln or situation, or was it a cascade thing, or what changed?

Dave Bittner: I think the emergence of ransomware was a big part of it. That becoming its own business, the attackers being able to take these vulnerabilities and use them in ways against companies that were truly potentially devastating to the organizations, the whole notion of reputational damage, which again, is connected to ransomware. It's just much -- and also, I guess, awareness among the general public that this is now a thing, that a cyberattack is a thing, because now, as opposed to even 10 years ago, everybody's online. Everybody has some connection to something online. There's no escaping it, and 10 years ago, I think it was a little easier to escape it if you wanted to.

Maria Varmazis: Yeah, and going back a little before 10 years ago, going back further than 10 years ago, Heartbleed was I think 2014, and that made a whole lot of people who had no idea what OpenSSL was, suddenly everybody had to become an expert on pretty quickly, especially if you had a business that did anything online, which is most businesses. Not that I want to say, hey, thanks everybody for naming vulnerabilities because it made it easier, but maybe that actually did help a little bit as much as we make fun of that kind of thing. It might have actually helped a little bit with awareness. Who knows? I'm just throwing that one out there. I'm going to get hate mail for that one. Anyway, looking back specifically within the last 10 years, if you had to choose maybe one vuln that was the most seismic with its arrival on the scene, is there one that you could point to?

Dave Bittner: Well, I think the most seismic was probably Log4Shell in terms of how broad and serious it was and how people responded to it. Log4Shell was, I would say, late 2021, I believe. It took advantage of what was a vulnerability in Log4J, which is this open-source logging framework. Again, not to get too deep in the weeds, but it was a very serious thing, and what made it so serious was how many devices it affected. It was just kind of this thing that was in everything, and so it meant that all these devices were vulnerable to it. I think Jen Easterly, when she was running CISA at the time, and she said it was one of the most serious that she'd ever seen in her career, if not the most serious. Even the Federal Trade Commission got involved to get companies to update so that they weren't vulnerable to this. It was kind of an all-hands-on-deck kind of moment in terms of seriousness. I think if you want to rewind the clock to when did a lot of this stuff start to become more broadly known throughout the non-IT world, I think you have to look at EternalBlue, which was, what, 2017 or so? EternalBlue was an exploit that allegedly came out of the U.S. National Security Agency.

Maria Varmazis: "Allegedly."

Dave Bittner: I think that's pretty settled at this point.

Maria Varmazis: Who ate Stuxnet? Okay. I think we're done.

Dave Bittner: Who knows? We'll never know. It's a mystery. This was a zero-day vulnerability affecting Windows systems, but it got acquired, found, whatever, by the shadow brokers who then famously used it in WannaCry, which used the EternalBlue exploit. WannaCry was a worm.

Maria Varmazis: And it made you "WannaCry." It sure did.

Dave Bittner: Yeah, and it was also used in the NotPetya attack, so if you want to talk about turning points, I think that certainly was one. Then, also, the notion that a tool that had been developed by our own intelligence community, air quotes, "the good guys," got turned against us, turned against the world.

Maria Varmazis: Yeah, and I remember before WannaCry, there was a lot of work being done, I think, within the IT world to try and help practitioners and, you know, IT execs even message the importance of this is why we need a patch. Sometimes it fell on deaf ears to the higher ups, so to speak. I feel like WannaCry kind of did a lot of that heavy lifting for people from that point. No security awareness campaign was ever going to be as effective as touching your hand on the hot stove like WannaCry was.

Dave Bittner: That's right. That's right. [Laughter] Yes, I'm laughing because as the father of a teenage son who goes through life touching hot stoves and that being the only way he can learn anything, you know, that resonates with me.

Maria Varmazis: I feel like, truly, it's a very human thing, very, very human.

Dave Bittner: Yes, absolutely.

Maria Varmazis: We warned you. We warned you. Oh, now you've got to learn the hard way. Oh, dang it. Now we've all learned the hard way. Yeah, but, I mean, WannaCry, and truly, if we bubble that up to EternalBlue in general, I remember there was also the geopolitical aspect. You touched on this a bit, as well, about, you know, governments having zero-day stockpiles and what does that mean? You know, what's the danger there? That's an ongoing conversation too.

Dave Bittner: Yeah, it was eye opening. You know, you mentioned Stuxnet before, and every now and then, one of these makes its way out. You know, it's a question that I've asked folks I interview, some of the researchers, you know, and particularly for a "Research Saturday" show, and it's usually a question I ask off the air, but it's --

Maria Varmazis: Yeah, it's when the interesting stuff comes out.

Dave Bittner: It's for my own curiosity and people, you know, feel more like they can share more, but I'll say, you know, how often do you come across something that has all the signs of being a piece of United States offensive tradeware or --

Maria Varmazis: Yeah, yeah, yeah.

Dave Bittner: They'll say, yeah, it happens. You know, there are -- yeah, of course it happens and --

Maria Varmazis: That's why espionage exists.

Dave Bittner: Right, yeah, right, exactly, so, you know, we joke about what we should call it, you know, "Eternal Eagle" or something like that. It's a very stereotypically American name, you know, "Righteous Eagle" or something like that. But anyway --

Maria Varmazis: Yeah, well, going back to touching the hot stove for a second also, there was what the sort of overall business community learned, I think, from WannaCry, but what would say, sort of, the takeaways for the security industry were from that?

Dave Bittner: For WannaCry?

Maria Varmazis: Yeah.

Dave Bittner: I think it a message of responsibility. Who's responsible for these things? In this case, this was affecting Windows systems, so to what degree is this Microsoft's responsibility to get patches out there to remediate this? Then to what degree is it the responsibility of agencies or researchers who stockpile these things, who know they exist, and rather than disclosing them, either sell them or keep them or save them for whatever use that they might be useful for in the future? What's the moral and ethical thing to do in a case like that?

Maria Varmazis: That is a question for much smarter people than us.

Dave Bittner: I concur.

Maria Varmazis: Yeah, it's a good question though. Yeah, so we've been talking about, you know, these paradigm shifts in vulnerabilities. Typically, when we talk about vulns, it's a software problem, but another huge paradigm shift in the last 10 years was when we saw vulns that were hardware problems, and I'm thinking of Spectre. That was massive. It's not just because I'm married to a hardware guy who works at one of these companies, although full disclosure, I am. I mean, that was -- that was a really -- that was, I remember, it felt actually kind of scary on a level of, oh, gosh, I didn't -- were we really thinking about that? I thought, you know, hardware issues were in the realm of the extraordinarily "nerdy security through obscurity." Why do we have to even think about that? We've got enough problems with software, and then came Spectre and Meltdown, yeah.

Dave Bittner: I remember when Spectre came out, my first thought was about a bug that affected Pentium processors back in, like, 1994 or 1995. There was some issue baked into certain Intel processors, Pentiums, that would return floating point results that were wrong. We kind of count on processors to get math right, right?

Maria Varmazis: It's a little bit important.

Dave Bittner: Yeah, yeah, and, you know, it was one of those things where it was a very rare bug. It probably wouldn't happen very often, but there was also a way that if you fed it this exact, very simple math problem, it would give you the wrong answer. It was very easy to illustrate, which made it easy to understand, which made it easy for people to be nervous about it. Intel did a recall and, you know, we all lived to tell the tale and live another day. That's what I was thinking of. Are all of these chips going to have to be recalled? Figure the difference between 2020 -- or I'm sorry, 1994 and 2016, when Spectre and Meltdown were, that's a lot more CPUs out there. That's a lot of hardware. What's going to happen?

Maria Varmazis: And how do we fix it? How do we, how do we easily patch that at scale?

Dave Bittner: Right, because hardware is hardware, unless, you know, talking about, like, I don't know, field-programmable gate arrays or something really down in the weeds. Your processor is your processor. Over time, all these assumptions had been baked into modern processor design and were tried and true, and suddenly, you discover a vulnerability. What does that mean? What it meant in the short term was that the patches made the processors slower.

Maria Varmazis: Because, well, you're adding something to it, right? I mean, yeah.

Dave Bittner: You're adding something to it, but you're also taking something away. Like, there was this -- my recollection, and I'm sure this is imperfect, but my recollection is that it had something to do with the predictive nature of how modern processors flow information through them, and some of the assumptions of what you could and couldn't do. That's where the vulnerability was, so you had to, basically, disable some of those predictive presumptions and that slowed things down. Now your server rack that yesterday you were selling that had these capabilities had those capabilities minus 20%, right? What do you do? [ Laughter ]

Maria Varmazis: It also, zooming out a little bit, really changed the conversation around what we sort of consider the arena of fair play for what needs to be actively monitored. Maybe that's the wrong phraseology there, but I don't remember the average bear talking about hardware vulns as much. Now, it's like, yep, that's part of what we got to be thinking about more regularly as opposed to that's a weird anomaly and just don't worry about that. Don't look in that corner. Yeah, it's just part of the field of play now, I suppose.

Dave Bittner: I guess, and I wonder, too, how much of it is just that we're at the point where so much of the low hanging fruit has been picked, so you're getting into more of these edge cases now, I guess. I know, you know, they'll take away our broadcasting licenses if we don't mention AI in this somewhere. [Laughter]

Maria Varmazis: Everybody drink.

Dave Bittner: Right, yeah, but I think AI has replenished that fruit.

Maria Varmazis: We'll be right back. [ Music ] We probably don't want to leave AI to the very end.

Dave Bittner: Okay.

Maria Varmazis: Yeah, although I kind of want to because there's so much that happened in the last 10 years before AI really shook things up. Yeah, I mean, AI is really adding -- I don't want to just say it's changing the paradigm. It's adding so much into the pipeline that it's just kind of hard to know what to do anymore. It's hard to know what needs to get prioritized. It's hard to know, you know, how to make sense of just the pure volume of it without saying, okay, AI is finding all these problems, let's let AI fix it, but where's the human in the loop on that? I mean, I think that's where we're all at right now.

Dave Bittner: I think so. To me, a big part of it for people who are fighting the good fight out there is it's a velocity issue that stuff is just coming at you so much faster. For me, like, you know, I -- back in the day, I used to enjoy playing first-person shooters like Marathon or Doom or, you know, the first-generation and the first version of Halo, right. I can't play them anymore. I can't watch my kids play them because they're too twitchy. They're too fast. I feel as though I was wired up, right? My first exposure to these was at a certain speed, and that speed is way faster than it used to be, and so, it's hard for me to adapt. The next generation coming up, that's all they know, so they can handle the speed. They don't think twice about it. I wonder if that's something that we're experiencing right now, where you have generations of researchers, people who came up with a certain cadence of patching, of remediation, all that's been thrown out the window because it's like -- what's the thing in Mario Kart where you get the speed boost, right? It's like that.

Maria Varmazis: The mushroom, yeah.

Dave Bittner: Yeah, and so how is it -- is it just the old timers who have to catch up, and then, the next generation is going to be fine with this new velocity because it's all they know, or as many people are saying, the only way to do this is machine versus machine, where the AI is the only thing that can be fast enough to keep up with the AI?

Maria Varmazis: Yeah, it almost makes me wonder if someone listening to this conversation a few years from now might go, "Oh, how quaint. They're talking about singular-named vulnerabilities. Who has time for that anymore?"

Dave Bittner: Right. They're talking about "humans patching," you know, "Oh, it -- what was that like? You people actually -- you had keyboards. You touched your computers? Eww, yuck."

Maria Varmazis: But also, that you had the time. Things were slow enough that you were looking at, you know, one or a group of things at a time as opposed to just like a torrent. I mean, I know vulns and patching. I mean, the reports lists are, you know, hundreds of pages long and, you know, nobody knows every single one that gets patched, but at the same time -- okay. That's not that's not true. The idea of, like, we're thinking about one specific vuln as opposed to, you know, tens of thousands at that the volume is just -- this is a totally different scale. We're going to be talking about things in scientific notation instead of just in dozens.

Dave Bittner: Yeah, like we thought we were in an age of automation. Like, hold on to the bar, because here comes a new age of automation that is driven by necessity.

Maria Varmazis: Okay. As we're talking about all of these huge paradigm shifts over the last 10 years, as we reflect on vulnerabilities, I would be remiss if I didn't also mention supply chain attacks. That was another huge shift in the last 10 years, and the biggie was SolarWinds. That's the one that, you know, is synonymous with supply chain issues. What's your recollection of how that went down when that started out?

Dave Bittner: Well, I think people were whistling past the graveyard, right, for a long time before that one landed, and it was proof that it wasn't a matter of if, it was a matter of when. SolarWinds hit and it was a massive supply chain impact all over the world, and, you know, affected real people. You know, they couldn't use their gas pumps. I remember in the southeast of the U.S. here, people were hoarding gas. So real world, regular people, repercussions of this, and it got everyone's attention and served as a, I guess, the proof-of-concept that supply chain vulnerabilities are a thing, and they need to be taken seriously. I think it kicked off a whole era of supply chain research, of things like SBOM initiatives, you know, software bills of materials. It triggered a bunch of scrutiny of open-source ecosystems.

Maria Varmazis: Oh, gosh, yeah.

Dave Bittner: Everybody started thinking about what's in my system that I don't know is in my system, right? It was the unknown unknowns.

Maria Varmazis: Thank you, Rumsfeld.

Dave Bittner: Yeah, because nobody's -- you know, nobody is -- I'm sorry, I shouldn't use absolutes. Most people are not creating software from whole cloth. They're using open-source packages. They're using -- because why not? Who has time to write everything?

Maria Varmazis: Yeah, it's just efficient. I mean, that's just how things are built now. I mean, that's how it's always been, really, is my understanding. I mean, you don't have to recreate everything from scratch. If there's a perfectly good library or something else that someone has done, you build on that. That's how we make progress.

Dave Bittner: Yeah, so if you have these dependencies, and you've been using this little nugget of open-source software for the past five years, and every time a new version comes, you wait a few days to see if there are any major issues, and then you just slide it into your own production environment, no problem, business as usual. Then all of a sudden, somebody can take that, put something bad in it, and without even realizing it, it slides into your environment. Now you've got a problem.

Maria Varmazis: Going back to AI, we're seeing such an influx of people vibe coding or trying to contribute to open-source projects using AI. What we're talking about right here is part of that conversation for why that is so dangerous. We have the idea of people making contributions to these extremely important open-source projects that a lot of our modern world is built on, but who's actually able to keep track of what's being contributed if it's being done through AI? Understandably, some open-source project maintainers are saying, no AI contributions whatsoever, but this is part of the reason why there's a lot of paranoia, understandably, about that, because who knows what's being introduced that way? It's wild seeing these things converge in real time, wild, scary, fun, interesting, I suppose.

Dave Bittner: It keeps a lot of people in business.

Maria Varmazis: Truly, and speaking of wild, fun, interesting, and all those things, a story that, honestly, before we started doing research for this, I had actually a little bit forgotten about, and I cannot believe I forgot about this. Bloomberg had this huge bombshell story about Supermicro, and that set off this sort of feeding frenzy to chase that one down, and it ends up that that one wasn't real. If you look for it, though, it's still there on Bloomberg.com.

Dave Bittner: Yeah.

Maria Varmazis: So it's -- I -- what the heck happened here?

Dave Bittner: Well, it was 2018, I believe, that Bloomberg published this story, and the allegation was that Chinese operatives had inserted these tiny, little hardware devices, little, tiny chips into Supermicro motherboards.

Maria Varmazis: Right, yep.

Dave Bittner: This was a supply chain compromise and a degree of sophistication we had not seen before.

Maria Varmazis: And a nightmare situation that, you know, all this physical hardware has been tacked onto all this incredibly important stuff on the motherboard, like, that's a nightmare.

Dave Bittner: Right, and these Supermicro servers are everywhere in the government, and, you know, very popular, a good brand, you know, all that. They got dragged into this. So everybody started looking for what's going on here, because if this is truly a hardware bug, well, the hardware exists. It's a real thing. It must be on the motherboard, and nobody found anything. There was never any public evidence that supported the central claims.

Maria Varmazis: Yeah, and I remember the initial reaction from a lot of folks in sort of the federal sector going, was just, how the heck did we miss this? Like, this is the kind of thing we're looking for. How on earth did we miss this, and how did Bloomberg find it first? The answer was that you guys didn't miss it. But that was a crazy cautionary tale. You know, everybody kind of, especially in the media sector, we all want to be the ones to report on the new, you know, hot vuln, because you want to be the place that breaks that story. That's a great way to be in front of stuff, but this one didn't exist, so it's a hard one.

Dave Bittner: Yeah, and do we think Bloomberg was acting in bad faith? I don't think so, but they got it very wrong. Whatever internal checks and balances they must have had failed them, in my opinion, and remarkably, what's been eight or nine years now, there's never been a serious retraction from Bloomberg. Like you said, the article is -- you can still go read it.

Maria Varmazis: So, Dave, this is -- we've only taken a very top line view of the many, many vulns over the last 10 years. Certainly, this is not an all-inclusive look. We could not possibly do that in even just a few hours. It would take us days. You know, it's been a good 10 years of you hearing these stories come and go. Some of them, some of these vulns have a pretty short life cycle. Some are ongoing still, like Log4Shell, for example. We're still dealing with these. What sticks with you as you look back on these?

Dave Bittner: Well, I think the thread that runs through all of this is this notion about having our assumptions challenged, right? Like we talked about at the beginning, people assumed, rightly assumed, that they had time to patch. People rightly assumed that software was trustworthy. People rightly assumed that hardware was trustworthy. They thought they had time to take care of things, and each of these events have challenged those assumptions. Who do you trust? To what degree do we have to scrutinize software or rely on other people to do that for us? To what degree could something be lurking in our hardware? We have all these assumptions, and ultimately, at some point, you have to trust things. You have to trust people. That's how the world works.

Maria Varmazis: I can hear a few people going, no, no, I won't.

Dave Bittner: Right, well --

Maria Varmazis: Well, there are a few folks that feel that way. That's true.

Dave Bittner: Yeah, yeah, and, you know, trust but verify, right? All that stuff. I mean, there's only so much you can do, but at some point, you have to believe something.

Maria Varmazis: Yeah, no, no, you're right, you're right.

Dave Bittner: As simple as when I flick the light switch, the light comes on, right? I think it's over the past decade, we've seen many of our assumptions fall away, and we've learned that things that we used to assume were true, now require an additional level of scrutiny, an additional level of care, double checking, all of those things.

Maria Varmazis: Assumptions we didn't even know we had in some cases, just, they were so baseline that we just --

Dave Bittner: Right.

Maria Varmazis: It was like breathing. We didn't even think about it, and now it's like, oh, we have to examine that as well. Oh, my God.

Dave Bittner: Right, and things have gotten so much more complex. Things are so much more interconnected than they used to be, and now we've thrown AI into the mix, which has put us all on turbo speed, and yeah, what a world.

Maria Varmazis: What a world. Well, we'll be here for the next 10 years, too, I'm sure, Dave, and --

Dave Bittner: I don't see it slowing down anytime soon, and it's always something interesting. That's the thing about this business, right? There's always there's always something to learn. There's always something new coming and surprises, and so it never gets old.

Maria Varmazis: That's an awesome way to end it. So Dave Bittner, host of the "CyberWire Daily," thank you very much for joining me today. [ Music ]

Dave Bittner: My thanks to Maria Varmazis for joining me as we continue looking back on a decade of cybersecurity stories that shaped the world we defend today. Thanks for listening to this "Special Edition of the CyberWire Daily's 10th Anniversary Series." We'll be back with more conversations exploring the people, the moments, and the milestones that have defined the past 10 years of cybersecurity. I'm Dave Bittner. Thanks for listening. [ Music ]