WireX BotNet with Justin Paine from Cloudflare.

Dave Bittner: [00:00:03] Hello everyone, and welcome to the CyberWire's Research Saturday. I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down threats and vulnerabilities, and solving some of the hard problems of protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us.

Dave Bittner: [00:00:23] I'd like to tell you a little bit about our sponsor, Cybrary, the people who know how to empower your security team. Cybrary is the learning and assessment tool of choice for IT and security teams at today's top companies. They deliver the kind of hands-on training fifty-five percent of enterprises say is the most important qualification when they're hiring. And once you hire, you want to retain. And Cybrary helps there too, because seventy percent of employees say professional development is a big reason for staying on board. Visit www.cybrary.it/teams, and see what they can do for your organization. Not only is it effective, it's affordable too - costing just about a twelfth of what legacy approaches to training would set you back. So contact Cybrary for a demo. That's www.cybrary.it/teams, and tell them CyberWire sent you.

Justin Paine: [00:01:20] We initially noticed this particularly botnet when we observed the particular user-agent string that was that was being used as part of this attack.

Dave Bittner: [00:01:30] That's Justin Paine. He's the Head of Trust & Safety at CloudFlare. The botnet he's describing is called WireX.

Justin Paine: [00:01:38] And what in particular caught our attention is - and when we see attacks roughly every three minutes, every day all day - but this particular botnet was different in that we were seeing this user-agent string, which is not usually something that would be so consistent. It was always 26 characters. It was always seemingly random, although there was some patterns in it, and that it was always alpha characters. But 26 characters every single time.

Dave Bittner: [00:02:10] And so, that draws your attention. What happens next?

Justin Paine: [00:02:15] Initially, quite frankly, we initially didn't look into it all that much, because the attacks were relatively low volume at the time. This was relatively early on as the botnet was starting to grow. But we had heard from some of our friends and other folks who worked in the industry at Akamai that they were seeing similar attacks against their customers, and the attacks that they were seeing against some of their customers were significantly larger and definitely causing some impact in some cases. So that certainly made us more interested, and I think, you know, the assumption is the person was probably using the botnet to target those customers at that time and hadn't quite turned it against, you know, our customers fully yet.

Dave Bittner: [00:03:05] So the notion is that, when it first came on your radar, that perhaps they were just testing it out?

Justin Paine: [00:03:11] That's what it seems like. Testing it out or spreading it too thin, perhaps early on, versus - relative to the size of the botnet. So they may have thought they had more power than they did in the early days, and they realized that they weren't taking down sites as well as they thought they should be able to, when they were spreading it between CloudFlare and Akamai customers. It looks like they may have then increased the size the botnet, in addition to kind of focusing it on a target versus dispersing it against a handful of targets.

Dave Bittner: [00:03:43] I see. So, take us through some of the details. How exactly does this botnet work?

Justin Paine: [00:03:48] So the botnet is consisted of Android devices, and that could be almost - largely cell phones, but also could be potentially tablets or other devices that run the Android operating system that would download one of these infected APK applications. That is typically going to be through the Google Play store. However, there are a variety of these third-party websites out there that, essentially, all they do is clone the Google Play Store. So they'll have the same applications, roughly the same type of design. And you can search for applications in their third-party store versus Google's first-party store.

Justin Paine: [00:04:34] And in both cases, those APKs from this particular developer - and he had multiple accounts, but it seems to be likely one developer - those APKs were malicious in that they included this additional code that could be activated based on commands sent from the command-and-control server, to either launch attacks against customers or launch attacks against website owners, or to do other things like click fraud, which is less flashy and impactful, but still obviously malicious in nature and does cause financial harm.

Dave Bittner: [00:05:17] So is this a case where the malicious code is sort of being grafted onto an existing app, or were these apps being spun up for the purpose of carrying this malicious code?

Justin Paine: [00:05:28] That's a great question. As best as we've been able to gather, it does not appear that these applications were compromised in some way. It doesn't seem like they were a legitimate application which then, maybe the developer had their account compromised, or something like that. This definitely does appear to be purely intentionally malicious, in that they wrote these applications and used icons in the various app stores to impersonate legitimate applications.

Justin Paine: [00:05:59] Sometimes the names were sort of similar; they weren't that close, because they don't - I think they wanted to avoid the attention of the legitimate app owner. So you're not going to make an app that looks just like YouTube, that has a name like YouTube, because that's going to be easily found and disabled. But they would use an icon that would look like the official YouTube app, for example, but the name would be significantly different. But yeah, the app would be intentionally designed in such a way that the command-and-control server would be called out to from the infected device and, depending on what that command-and-control server sent, it would take particular actions, whether it's attack a website or conduct this click fraud process.

Dave Bittner: [00:06:47] And to the users, the people who have installed these malicious apps, would they notice anything going on on their device, or was it happening all in the background?

Justin Paine: [00:06:57] Most of it was in the background. So that the possible side effects that the - that someone infected may have noticed would have been, you know, their battery life may have significantly drained, the device itself may have actually become warm to the touch because it was the processor, everything else on the phone was working pretty hard to push out this traffic as part of the DDoS attacks.

Justin Paine: [00:07:23] And if they were on a cell network, while the bandwidth wasn't really the core of how the attacks were were working, it is possible that they may have been seeing significant spikes in bandwidth on their cell plan bill that they received at the end of the month, for example. But this particular type of botnet wasn't focused on bandwidth exhaustion. It was more application-level resource exhaustion.

Dave Bittner: [00:07:52] And so, what were the types of attacks that you saw being implemented by this particular botnet?

Justin Paine: [00:08:00] The WireX botnet launched volumetric DDoS application layer attacks. And what that means is they're not trying to exhaust the bandwidth to whoever it is they're trying to attack. So they're not trying to fill up the pipes with so much bandwidth consumption that you can't get to whoever the victim website is. Instead, what they try and do is each one of these infected devices looks like a legitimate real web browser with a real human behind it.

Justin Paine: [00:08:30] So the way that the botnet was implemented on the Android device - the browser that was accessing this website repeatably had all the functionality of a real browser. It had JavaScript functionality, it had local storage, it had cookie handling, it had everything that looked like a real user. And it would repeatedly access particular resources on a given website to try and just eat up CPU on the server.

Justin Paine: [00:08:58] And it did this in a number of ways. So, not only did it access particular parts of a website that may be resource intensive, like a search page where it needs to go search a database and retrieve it a result and then display it. That takes a lot of resources on the server side. If you're doing that thousands of times a second, that may take down the server or significantly slow it.

Justin Paine: [00:09:23] In addition, they also used HTTPS. So it wasn't a standard HTTP plaintext request to the server each time. These were largely encrypted HTTPS connections that would further use additional resources on the server side of things as well. So each one of those packets has to be handled by the server and decrypted and do all the normal - the handshake that happens as part of a HTTPS request.

Dave Bittner: [00:09:53] What does it seem like they were up to? Was there a ransom component to this?

Justin Paine: [00:09:57] We did see a handful of cases - and this was specifically on Akamai's side of things - they saw a handful of cases where a ransom email was sent to some of the organizations that were being attacked. Although it seemed it wasn't - most of the websites that were attacked didn't receive a ransom email. And the email that some of these folks did receive - it wasn't sent to a standard, functional email address. Like, it wasn't sent to, you know, "admin@" or "noc@" or anything like that. Most of these ransom complaints that we do see from these ransom DDoS trends that have been going on lately - they use these functional email accounts, whether it's "support" or "noc" or "admin" or things that are likely to exist.

Justin Paine: [00:10:48] In these cases, they emailed what appeared to be relatively random email addresses at the places that they were attacking. So it seemed like maybe they were experimenting with this and haven't really fully decided to do it, or something potentially like that.

Dave Bittner: [00:11:06] So,the distribution of the malware was significant.

Justin Paine: [00:11:12] Yes. We don't have exact volume numbers in terms of how many downloads. Unfortunately, Google was actually too quick, which is a good thing to have - that's a good problem. They very quickly, once notified, they very quickly began to take action and pull these applications out of the app store, the Google Play Store. So we weren't able to grab download counts before we noticed that they were already disappearing because Google was doing a fantastic job.

Justin Paine: [00:11:43] We know that the infected apps were in the Play Store for a little under a month. So roughly thirty days and, you know, I would have to only guess at the possible download counts, but yeah, we saw, at peak, 120,000 unique IPs that were participating in attacks. So that doesn't necessarily mean 120,000 infected devices though, because these were largely mobile devices.

Justin Paine: [00:12:16] So the particularly unique challenge with a botnet that is comprised of mobile devices is some of them are on the cell network. And a mobile device is going to hop from cell tower to cell tower. And each time it does that, it's going to get a different IP address. And if it's participating in this, you know, in an attack. You're going to see one device potentially coming - using, you know, depending on how far they travel - five, ten, twenty, thirty IPs. So we saw around a maximum of 120,000 unique IPs, but it's really difficult to gauge how many unique devices are behind those IPs.

Dave Bittner: [00:13:04] Take us through the process of how you actually track down the software.

Justin Paine: [00:13:08] When we began the process, we had identified this was an interesting botnet. We had spoken with some folks at Akamai that identified a similar botnet. We started digging into the the log files that we had available of what signatures does an attack from this botnet really show. And that led us to some particular traits of the attack. A single packet that would be sent from one of these devices that was infected would have a particular tell in it, which I can't unfortunately elaborate on. But there were there would be a particular tell from that that request that would essentially say, hey, I got infected from here. Which was really convenient because, you know, it basically said this is the infected Android application that caused this phone, this device, to participate in the attack.

Justin Paine: [00:14:04] Which, once we discovered that, everyone kind of had that light bulb moment of, is that really the application that may have led to this infection? And we did a bit of research and searching, and in most cases the apps had already been pulled from the Google Play Store, but some of these third-party websites that duplicate a lot of the Google Play content still had several of them available online and able to be downloaded.

Justin Paine: [00:14:33] And that's where we luckily were able to grab one of the APKs that we believed to potentially be infected, and we decompiled that APK and were able to identify what appeared to be a call out to some external domain. Which isn't necessarily malicious in nature, but based on the traits that we are seeing of this APK possibly being involved in a botnet, the assumption was that there was likely to be a command-and-control server of some sort. So this APK calling out to some external domain was a pretty good indicator that maybe we were on the right track.

Justin Paine: [00:15:20] And that's where we did ultimately find the domain that was the command-and-control server for this particular botnet. Once we had identified that, we were we were able to just start querying different subdomains that were available for that domain to see what it might output. And ultimately we found one particular subdomain that, when queried - this is, you know, you load it in your browser, you load it at the command line something like that - it would return what domain should be attacked by the botnet, in each request.

Justin Paine: [00:16:01] So that gave us an inside view of, okay, who's currently being attacked by these devices? Which was a pretty significant breakthrough for us in terms of, okay, now we know who's being attacked, and - if they happen to be an Akamai or CloudFlare customer - we could very easily attribute that to this exact botnet versus potentially some other attack that may, purely by coincidence, happen to target that customer.

Dave Bittner: [00:16:29] Is the botnet still up and running?

Justin Paine: [00:16:31] It is largely dismantled, and by that I mean there are still efforts - law enforcement is engaged and is actively working on the case, and I can't elaborate on that point, but the vast majority of the functionality of this botnet that has been neutralized. The Google Play Store has removed the apps that would cause an infection. Most of the applications from these third-party websites have also - we've gone through the process of submitting, effectively, abuse reports, and kind of complaints about those applications on those sites and they've been largely taken down. And the command-and-control server that is hardcoded into any applications that may still be out there. So, even if a device happens to be still infected, the command-and-control server itself is also dealt with at this point. So a device that may still be infected won't get any commands to launch an attack. So it should be largely neutralized at this point.

Dave Bittner: [00:17:33] Is there anything you can share with us in terms of attribution, of where it was where it was coming from, or who might be responsible?

Justin Paine: [00:17:41] I wish I could say with confidence, you know, that we knew who this was and where they were located. However, that is remarkably difficult to say with any real confidence..

Dave Bittner: [00:17:53] Sure.

Justin Paine: [00:17:54] Unfortunately, it would be a guess, at most, and I'd rather not speculate.

Dave Bittner: [00:18:00] One of the points that you make in the report is the success in sharing information among many of the players among those who sort of sorted this out. Can you take us through that? How that was a real sort of force multiplier for you?

Justin Paine: [00:18:17] So most of the companies that were involved in this particular process - many of us are in the same industry. So CloudFlare and Akamai are - we provide very similar types of application delivery platforms. Some of the other organizations who were involved, such as Flashpoint and RiskIQ, they don't directly have an obvious crossover in terms of where our companies live.

Justin Paine: [00:18:42] But that being said, these are organizations who have been working together for quite some time to address significant events that impact the Internet. And by that I mean these groups largely began to get together consistently when Mirai began to be a pretty significant problem, middle of last year, you know, middle to late of last year. These companies, at that time this was a significant problem that everyone saw and it wasn't, you know, that's my network or that's your network, that's not my problem.

Justin Paine: [00:19:17] These were Internet-scale concerns that really drew together individuals from across companies and, you know, some of them may normally be competitors, but, at the end of the day, you know, you're not going to be a competitor if the Internet is down. So we've learned to become friends over time, but, you know, many of these relationships initially began as it made the most sense to work together - to try and pool resources, pool intelligence, I mean, intelligence of, you know, very smart employees, but also pooling information about the attacks we were seeing in a way that, you know, still strongly holds true to our commitment of privacy to our customers, but there's still ways you can share anonymized information that may ultimately still be useful, but doesn't necessarily impact the privacy of the company's customer base.

Dave Bittner: [00:20:12] Would a standard antivirus installation detect this?

Justin Paine: [00:20:16] At the time that we were looking at these particular APK files, some of the antivirus vendors were identifying these particular APKs as being malicious. So there was certain indicators within it within the application code that, you know, these antivirus vendors had a sense that something strange was going on. Most of them didn't pinpoint the specific DDoS attack functionality that were in the applications. They were largely identifying the click fraud related behavior that seemed to be kind of the initial purpose of these applications.

Dave Bittner: [00:20:59] In terms of general advice for people who want to protect their devices from inadvertently becoming part of a botnet, what kinds of things would you want to share?

Justin Paine: [00:21:08] I would say it's a good rule of thumb to stick to only acquiring applications from the first-party stores for your particular platform. So the Google Play Store if you're on some type of Android device. The Apple App Store if you're on an iOS device. They have significant resources dedicated to scanning applications that get submitted to those app stores to ensure that they are reasonably safe.

Justin Paine: [00:21:36] Sometimes things still do slip through the cracks, but they have far more resources dedicated to securing platforms versus these third-party websites who appear to clone the legitimate app stores. However, there is no independent confirmation that the application here downloading is really the application you think you're downloading. There's no fingerprint on the application that shows it hasn't been modified. So you really are taking a significant risk if you're downloading from a third-party app store.

Dave Bittner: [00:22:09] Yeah, stay out of bad neighborhoods, right?

Justin Paine: [00:22:10] Absolutely. You wouldn't take candy from a stranger, you shouldn't be taking third-party applications from a stranger, either.

Dave Bittner: [00:22:23] Our thanks to Justin Payne from CloudFlare for joining us. You can find the research about the WireX botnet on CloudFlare's website. It's in their blog section.

Dave Bittner: [00:22:32] And thanks again to our sponsor, Cybrary, for making this edition of Research Saturday possible. Visit www.cybrary.it/teams, and see what they can do for your organization.

Dave Bittner: [00:22:45] Don't forget to check out our CyberWire Daily News Brief and podcast, along with interviews, our glossary, and more on our website, thecyberwire.com.

Dave Bittner: [00:22:53] The CyberWire Research Saturday is produced by Pratt Street Media. Our coordinating producer is Jennifer Eiben. Editor is John Petrik. Technical editor is Chris Russell. Executive editor is Peter Kilpe. And I'm Dave Bittner. Thanks for listening.