Bugs and working from home.

Dave Bittner: Hello, everyone. This is Dave Bittner, co-founder and host of CyberWire. Before we get started today, I have an exciting announcement. CyberWire is growing. We're thrilled to announce that CyberWire and CyberVista, an industry leader in data-driven cybersecurity training, are joining forces to form parent company N2K Networks, the world's first news to knowledge network. One of the insights we gained about our business since we launched back in 2016 is that you aren't just listening to CyberWire to keep up on the latest news. You're listening to learn. And over time, you've told us that we've become a critical part of your professional lives, a tool that helps you do your job better. That's news to knowledge, and we're excited to lean in on this idea and do more than ever before.

Dave Bittner: So CyberWire and CyberVista are coming together to connect news to knowledge, one continuous spectrum of situational awareness and learning. The union creates powerful new opportunities for professionals to keep abreast of the latest developments in their industry, climb the knowledge curve quickly and stay ahead in a rapidly changing world. As always, you can continue to count on us at CyberWire to deliver the world-class content you rely on. It's only getting better from here. And if you're new to CyberWire, welcome. Be sure to check out our other shows and partner content. We have more than 20 different shows on our network, and there's something here for everyone. You can find them all on our website, cyberwire.com/podcasts. Thank you for being a valued member of our CyberWire community. And now, back to your regularly scheduled programming. 

Dave Bittner: Hello, everyone, and welcome to the CyberWire's "Research Saturday." I'm Dave Bittner, and this is our weekly conversation with researchers and analysts, tracking down the threats and vulnerabilities, solving some of the hard problems of protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us. 

Federico Kirschbaum: Our research team was eager to know what the most common routers were actually running, and we're - when we are talking about routers, we're talking about the Wi-Fi router you have currently in your home. 

Dave Bittner: That's Federico Kirschbaum, CEO and co-founder at Faraday. The research we're discussing today is titled "A vulnerability in Realtek's SDK for eCos OS: pwning thousands of routers." 

Federico Kirschbaum: And that kind of got kicked off because, specifically in cybersecurity, we know that IoT devices - sometimes they're built without security in mind. But we never thought it would be this bad. And the idea of researching just a random device actually turned out - with this question is, who makes your home router because we found out that your vendors are using a lot of vendors to build that? And sometimes that's unclear, and that's not really easy to find out. And that affects a lot of brands that you didn't even know they could have a vulnerability. So yeah, it began as a weekend project. 

Dave Bittner: So I guess a lot of the routers that you could buy are OEM'd from, you know, a central source. You know, they make this hardware available for other folks to put their brands on. Can you walk us through how exactly you all centered on this particular provider here and how you got to this high-severity vulnerability? 

Federico Kirschbaum: So when we decided to do this research, our team decided to go to - we are based in Argentina. And in Latin America or Amazon, it's called Mercado Libre. And Mercado Libre basically provides a list of the most-sold devices in the region. And we found out that there were three brands, different brands, that were the most-selling ones. And by the way, they were the cheapest. And we bought them, all three. Little did we know these three different brands were actually the same device. They were actually using the same hardware, and they were using the same SDK. That means that when we found something - and so, like, OK, this is interesting, an interesting finding, and we jumped to the next device, the device was almost the same. 

Federico Kirschbaum: And that was kind of the moment we realized, if we find something here, we might be affecting thousands of brands. And the process basically involved opening up, understanding how it works, understanding the supply chain on how this specific vendor distributed their hardware and software to other vendors. And then I think - besides the problem we found and how we were able to take advantage of that flaw, I think it all took us by surprise, the amount of hardware that is being sold under different brands. And it's actually the same exact hardware and almost exact same software. So I think that's one of the key things of the research, of understanding the supply chain effect that has, you know, this idea of having hundreds of thousands of routers on the shelves. 

Dave Bittner: Well, let's talk about the vulnerability itself. Can you walk us through that? What exactly are we talking about here? 

Federico Kirschbaum: To put it lightly, it's basically a memory corruption. So that memory corruption allows somebody from the outside, from the internet, without credentials, without any knowledge about how to access the device, to manipulate device memory and execute arbitrary comments or insert, backdoor or reroute traffic. So basically, it's the worst type of vulnerability you can have because you can have no control on limiting the impact of that and you don't have an option that you can turn off, so you are not affected. And this vulnerability came by default. To be more precise, the vulnerability was happening in how the router creates mappings when you're using voice over IP. So the device, it's basically listening all the time for specific SIP packages. And when the device detects a SIP package, it's going to trigger a specific configuration to create that mapping between the outside call and the computers that are within the network. The main problem is that their implementation on how to process that specific package is not really well done. And by just sending one package, it would allow us to, first of all, crash the device. But that's not interesting. Our routers crash all the time by default. But it allowed us to start thinking on - once you get into a router, what would be the next step for an attacker? And mainly in this days, where we're working from home and you're accessing corporate from here, the idea of having a no-interaction, one-packet exploit, it was kind of what got us going. And that was kind of the first things that we found on these Realtek devices. 

Dave Bittner: So explain to me - if I'm the person who's trying to infiltrate this router to take it over, what exactly would I have to do here? And once I do those things, what kind of access do I have? 

Federico Kirschbaum: So normally, when we talk about routers and vulnerabilities, normally they are affected by services they expose. So by - the vast majority of these devices, when you buy them, when you connect them to the internet, they are not exposing anything. They are not exposing an admin interface. They're not exposing any other services because, normally, their customers or clients are on the other side of the network. They're on the landside. And if you are able to find a bug in this routers, you normally cannot attack them through the internet. And that's the key difference from our finding. These routers were listening to all packages, but basically processing a specific type of package. So if you're an attacker and you're able to send a malformed IP package to a router, that router, when it processes the package, is going to be executing attacker's code. And that specific piece of code will, first of all, allow the attacker to turn on remote admin device, for example, Telnet, which is - it's disabled by default. So first of all, he's able to access the device where there was no services. And that gives them admin access to the device. And that's quite an accomplishment because you can change DNS, so you can reroute the client's traffic to somewhere else. And you can start thinking. But in front - having a shell, it's just one of the first steps. And our research team found that the device itself had the ability to write arbitrary piece of data in the flash. And they were able to write a specific comment that allowed us to create tunnels, so basically create a mapping between an internal house and an external port, but also to port scan - just the idea of showcasing the ability of an attacker of, what happens when they have control? And when they have control, they can modify any option. They can create new options or new tools. And they are able to find more targets to attack. So this would be kind of arriving to a beach and just trying to invade the most part of the network. So these are awesome way to get in. And to be more precise, the specific type of vulnerability was not easy to find and to detect because the service, it's embedded in the networking stack of the device. So the only way to know if that device was vulnerable was actually attacking it. And since it's UDP, you could - if you're somebody who wants to have a lot of access, can easily spoof that UDP package and basically mass-scan the internet. And on the beginning, we thought it was just an issue for a couple of routers. Yes, it was an OEM. Yes, it was Realtek. But when we started, like, digging more deeply before our lecture at DEF CON, we found out that some devices in some countries - they were configured differently, and they were exposed in some versions. So we were able to understand how much and how many there were on the internet. 

Federico Kirschbaum: And just on that non-default setup was around 70,000 devices from all over the planet. And we're talking about the more different configuration we saw from all the routers that we tested. So if you're an attacker and you're thinking on how to attack next, you're always looking for the easy way in. And having access to the most important device on your network that can alter how your computer behaves - it's something quite rewarding from the hacker perspective and can give the attacker a perspective that is difficult to detect, it's easy to gain access to other computers once you're in and you get a perfect scenario for them listening, sniffing or just modifying traffic. 

Dave Bittner: Now, how has Realtek responded to this? Is there a patch available? 

Federico Kirschbaum: Yes. So we - when we discovered this finding, at that time, we didn't understood how bad it was. We actually submitted, like, four CVs. And the team at Realtek was quite responsive. We had the classic conversation between a vendor and security researchers. For them, it was not severe. For us, it was. The only way to prove it is by exploiting it and showcasing how bad it was. And with a little back-and-forth, they were able to submit a patch. And this is the - I would say, one of the interesting things. If they supplied the patch, also, the vendors who are using that SDK need to apply that patch. And when we're talking about other vendors, we're talking about the customers of Realtek because at the end of the day, they are the ones who purchase the hardware and the accompanying SDK to make their own router. And that small difference, it's, I think, the main problem we would have with IoT. Once you build it... 

Dave Bittner: Right. 

Federico Kirschbaum: ...No one will maintain it or have that in mind. That was one of the things. So yes, they supplied the patch. Many vendors applied it. But I think many others will never do it. And Realtek was kind enough to provide us with a bug bounty. Do you want to know how much was the bug bounty? 

Dave Bittner: (Laughter) Yes. 

Federico Kirschbaum: So we were honored, and they paid us 1,000 Taiwanese dollars. And on the beginning, you're like, OK, that ain't bad... 

Dave Bittner: Yeah. 

Federico Kirschbaum: ...Unless you do the conversion rate, and you go to $34 for... 

Dave Bittner: Oh, my. 

Federico Kirschbaum: ...This bug. The good thing is that we can - we could pay 75% of the router we bought for the research. 

Dave Bittner: (Laughter) So, I mean, just to be clear here, I mean, so Realtek is an OEM supplier that - they - you inform them of this issue. They come up with a patch for it. But then, as you mentioned, I mean, it's up to the folks that they provide this hardware to, the other people who put their names on this OEM product, to have - to apply that patch. But I suppose the bigger issue is that there are tens of thousands of these out there that are likely never going to be patched. The users probably aren't aware that they have an issue, and there's no way to automatically push a patch to them, right? 

Federico Kirschbaum: Exactly, because the patch that Realtek provided - it's not exact in fashion for their firmware that other vendors are using. And again, we are using the most cheap router you can find. It's something that when you see it, it's like, I wouldn't buy that. But that's the problem. If you or myself would buy a router, we wouldn't buy those. But what we found out is it doesn't matter. And it's the same pricing for several brands. So it could be double, triple the price. But actually, the hardware and what's running inside, it's the same. 

Federico Kirschbaum: So we're talking about people who may have the option of buying a router and they decided to buy the cheapest router, or we're talking about ISPs that bought routers by a thousand, and they applied their branding on top. So, for example, Brazil - it's a country that it got really affected by this vulnerability. And the differences between the original firmware and theirs - normally it's customization around specific config on their provider or just branding logos. And I'm not sure those devices are available to apply the patch directly. They need an overview from the vendor who customized that SDK because at the end of the day, Realtek is providing the hardware, most of the connectivity around their system of a chip and they are giving you the code so you don't have to rewrite, you know, the web server, the WPA2 config. But there is another part of that that's made by the vendor. And I think as a security researcher, gives me a lot of interest in understanding how supply chain works and how Realtek is going to take care of their customers on their behalf. And I think that's the most challenging thing this problem has. 

Dave Bittner: Yeah. So suppose that I'm the person at an organization who's responsible for security, and I have a whole lot of my employees who are working from home. Maybe they're using their own devices here or, you know, going through, as you say, their ISP. Is there a way that I can scan or have them check to see if they're using this hardware? 

Federico Kirschbaum: Yes, there is a way. And I'll put two scenarios for everyone's states. The first scenario, which I think is the most friendly one, would be sending a piece of code to your employee's computer and test what router is running at their home. That could be basically found out through their Mac address or just capturing the admin interface from the router. That could be the easiest way. But you would need collaboration, or you would need some sort of admin access to their computers so you can execute a piece of code that can give you an answer. That would be a great idea to understand how many of my employees are running a router that might have this vulnerability or maybe others, right? 

Federico Kirschbaum: If we go to the specifics of the vulnerability that our research team found, we have GitHub, where we supply all the information on the OS that it's running this specific Realtek device. And we also provide a piece of code that is a proof of concept that is - unless you're using a specific device that is the one that we use, it's just going to crash that device. And that would be a test as well - basically getting all your users from the VPN concentrator - IP addresses - and start one by one just, you know, crashing their routers. If they lost internet, you know, they're vulnerable. That would be the easiest way. I wouldn't say it's the most political one, but... 

Dave Bittner: Right, right. 

Federico Kirschbaum: Definitely it's... 

Dave Bittner: But it works. 

Federico Kirschbaum: It works. Exactly. 

Dave Bittner: Yeah, yeah. 

Federico Kirschbaum: And if you're in a hurry, that's a problem. But to be honest, the main problem happens afterwards. And it's - if an employee is vulnerable, you have two options. You can patch, or you can try patching, or you need to submit a new router. And that's kind of the difference. The companies that we try to protect end up taking more responsibilities on the hardware that the employee has in their home, what type of setup, what kind of investment. And sometimes that's a little bit overseen. 

Dave Bittner: It's a really good point. I mean, you know, we spend hundreds or thousands of dollars on setting up our employees with laptops or computers and so on and so forth. And here that could - there's a real - there's a high possibility of compromise through an inexpensive router that, you know, that is sitting there, doing its job and not drawing attention to itself but could be the real problem here. 

Federico Kirschbaum: Indeed. And I think there is a second takeaway from our research. The type of vulnerability that was found was not interesting. And what I mean it was not interesting - I mean this - it's the exposure of a beginner failure. They were using insecure C functions. They were using just a string copy in the wrong place in the wrong time. The problem is not that. The problem is that it's 2022, and this problem has been there since the '90s. We have the tooling. We have things that you can detect on the pipeline when you're building. 

Federico Kirschbaum: So it's like having a misspelling in the front cover of a major newspaper. There are things in control that are there that - A, they're not being respected, or B, they're being ignored, which is even worse. So that's the second takeaway that I'd like to share with you, Dave - this idea of the problems that we face from security are not new. And that's the main problem. It still affects millions of devices, and it's because somebody decided to ignore a warning from, you know, their static security analysis tool. And that's what worries me. I wish problems were a little bit more complicated sometimes. And having this scenario creates a lot of doubts on all devices that I currently own. 

Dave Bittner: Our thanks to Federico Kirschbaum from Faraday Security for joining us. The research is titled "A Vulnerability in Realtek's SDK for eCos OS: Pwning Thousands of Routers." We'll have a link in the show notes. 

Dave Bittner: The CyberWire podcast is proudly produced in Maryland out of the startup studios of DataTribe, where they're co-building the next generation of cybersecurity teams and technologies. Our amazing CyberWire team is Rachel Gelfand, Liz Irvin, Elliott Peltzman, Tre Hester, Brandon Karpf, Eliana White, Puru Prakash, Justin Sabie, Tim Nodar, Joe Carrigan, Carole Theriault, Ben Yelin, Nick Veliky, Gina Johnson, Bennett Moe, Chris Russell, John Petrik, Jennifer Eiben, Rick Howard, Peter Kilpe and I'm Dave Bittner. Thanks for listening. We'll see you back here next week.