Research Saturday 2.17.18
Ep 23 | 2.17.18

The uncanny HEX men.


Dave Bittner: [00:00:03] Hello everyone and welcome to the CyberWire's Research Saturday, presented by the Hewlett Foundation's Cyber Initiative. I'm Dave Bittner and this is our weekly conversation with researchers and analysts tracking down threats and vulnerabilities and solving some of the hard problems of protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us.

Dave Bittner: [00:00:26] And now a moment to tell you about our sponsor the Hewlett Foundation Cyber Initiative. While government and industry focus on the latest cyber threats we still need more institutions and individuals who take a longer view. They're the people who are helping to create the norms and policies that will keep us all safe in cyberspace. The Cyber Initiative supports a cyber policy field that offers thoughtful solutions to complex challenges for the benefit of societies around the world. Learn more at

Daniel Goldberg: [00:01:03] We're constantly looking at different sensors spread all over the Internet. We take a look and we have different like statistics modules and filters that look at and try and find something interesting and the Hex-men came up as this really scripted attack that was dropping something that was completely unknown.

Dave Bittner: [00:01:20] That's Daniel Goldberg a security researcher at GuardiCore Labs. The research we're discussing today is called "Beware the Hex-men," and it tracks multiple attack campaigns conducted by a Chinese threat actor. The GuardiCore team identified three attack variants that they named Hex, Hanako and Taylor. The target SQL Servers.

Daniel Goldberg: [00:01:40] So it goes into our queue to take a look and we take it open and we realize that the attack is kind of interesting kind of different from what we've seen before, attacking SQL Servers. OK so it's like OK let's start taking it let's start taking it apart figuring out what's going on. We're like OK, one attack one binary, cyber attack, second binary, and so forth. At some point after like 30 different malware samples, we're like OK we've got something serious on our hands. From that point on and then it's started like you start unraveling the thread of like trying to find different things putting it together. Then we found like three different attack campaigns all connected through the same infrastructure.

Dave Bittner: [00:02:18] We have the three different attack campaigns that are part of what you're calling the Hex-men trio. You've got Hex, Taylor and Hanako. Shall we walk through each of those individually?

Ofri Ziv: [00:02:28] Just before Daniel starts to walk through the three scenarios, one more word about the way that we first discovered the Hex-men.

Dave Bittner: [00:02:39] That's Ofri Ziv, he's the Vice President of Research, and Head of GuardiCore Labs.

Ofri Ziv: [00:02:43] We have a system based on the GuardiCore technology. Basically, this network of sensors, of GuardiCore sensors is based on the GuardiCore deception technology, which we just, you know expose to the Internet. This allows us to see a lot of malicious Internet activity. Together with a lot of knowns like Conficker worms and the WannaCry attacks. Next to those guys, you have the new and maybe sometimes more interesting campaigns and Hex-men is exactly this example, where this Internet-facing system allows us to detect this new thing, a new type of malware and after the system tracks those attackers and generate this malicious incident for us, then we take a deeper look into it and try to do the further investigation to do something like connect the three campaigns into a single under one roof. This is something that need the extra expertise, and this is something that Daniel can elaborate on.

Dave Bittner: [00:03:59] All right Daniel why don't you take us through. We've got these three components here what did you discover?

Daniel Goldberg: [00:04:04] Yes, so I'm actually going to go like reverse chronologically and I'm going to start with talking about Taylor and Hanako and then work back to Hex.

Dave Bittner: [00:04:11] OK.

Daniel Goldberg: [00:04:12] That's not the order we discovered them. So Taylor is at some point the interesting widespread botnet campaign you could read about. In fact, Taylor was partially discovered previously not by us, as a worm and is spreading through SQL Servers with attacks based from Linux and Windows machines. That seems to have a keylogger and backdoor component. And the reason for name Taylor is very simple. As it downloads the backdoor by hiding the data inside an image of Taylor Swift.

Daniel Goldberg: [00:04:45] This is simple and it works perfectly well because most sensors see OK you're downloading a JPG. It's all fine and only when you open it up the image and you're like OK this is full of junk and there's a lot of binary code here that contains the keylogger. The other one is Hanako. Hanako is a pretty like big DDoS, I'm going to say, campaign that attacks different machines. This is probably like if you had to think of the archetypal botnet that this would probably be it. It attacks Windows and Linux machines that tries to brute force them. My skill in the MS SQL Servers. The interesting part there is mostly that it's very similar and yet it's unique. These are the less interesting going will reach the Hex variant where we started from. And the interesting part there is that it seems to focus on Windows MS SQL Servers which are really widespread a lot more than you would think for a not a free offering, and they are incredibly varied. We've counted over 300 different sub-variants in this attack which means the attackers are pretty much going, OK, we're probably going to get caught somewhere, so we're going to make sure every attack looks slightly different.

Daniel Goldberg: [00:05:58] So if you're going to be like this analyst looking at the list of indicators of compromises, IOCs, you're going to be like, OK, another small one, another small one, but if you look at the attack flow, which was what we were looking at, because we are starting, as we said from the disruption service then we're seeing okay these are all identical, they're just dropping slightly different binaries.

Dave Bittner: [00:06:20] Well, take us through that attack flow.

Daniel Goldberg: [00:06:22] The attack flow is really long. It's one of the longest brute force attacks I've seen. It tries out by starting to connect to an SQL Server, and once it's in, what it does is make sure they can hide from auditing by turning off every possible audit method and then is going to try to actually attack using a variety of different methods. So the thing is there's usually like three well known attack methods for SQL Servers. The most famous one is just loading a plugin that lets you execute shell commands. Attackers don't necessarily avoid one or the other, they just use each of them in a way that tries to go under the radar and uses each attack for the best it can.

Daniel Goldberg: [00:07:06] For example they write files to disk, not by outputting to shell commands long strings, but they use a sub-database, Access database. Classic ones that are also installed with the SQL Server to write files to disk. And this looks very legitimate. This is not a shell command piping data around the system that some monitoring tool could see, it's more like OK SQL Server wrote a file to disk. Who the hell knows what skills server is doing? The same thing is used to change registry settings and security settings instead of, instead of again, OK let's say turn off the noisy matter, we're going to use WMI, Windows Management Instrumentation classes, and we're going to configure them so everything is turned off. We're going to make sure that the security settings allow us to change everything and we're going to do again everything through the SQL Server process. It's only the really last moment when everything is ready, that they use shell commands to actually run their tools. At that point from a standard security software perspective, all SQL Server is doing is executing a file that's already installed properly on disk, that is configured properly to be permissible to SQL Server to execute and everything looks OK.

Dave Bittner: [00:08:20] And part of what it's doing, it's disabling antivirus software, yes?

Daniel Goldberg: [00:08:26] Yeah, that's in the second stage after it's starting to execute its own code and not through the SQL Server is to methodically kill antivirus software. And it's very methodical about this. And by trying, like, I think it was about around 20 different products both, known antiviruses that you might even have tried at some point and really unique ones. It explicitly tries to kill an antivirus called BullGuard, which is aimed at the gamer crowd, which I didn't know was a real antivirus market segment, but it knows about it and it also tries to kill it.

Dave Bittner: [00:09:01] Now it's interesting in your research here, that you've broken the attack infrastructure into three different classes, of steps, basically. You've got to scanning, the attacking, and the initial implant.

Daniel Goldberg: [00:09:13] Yeah. So you're right and the split is something we're starting to see a lot more in botnets the last few years. We've also seen it last year when we saw a botnet we called the Bondnet, that was also an attacking service.

Daniel Goldberg: [00:09:28] The idea is, you don't see, as a defender, a lot of traffic to any particular server. You have breached your scan device specific server, but you're breached by a completely different server. It's harder for you as a defender to connect the two events, and after the breach the communication to download the further Trojan stages happened towards the third server. As a defender, it's very hard for you to piece together what's going on.

Dave Bittner: [00:09:53] I see.

Daniel Goldberg: [00:09:53] That's one strong advantage. The second is, all this infrastructure is hosted on previously compromised machines, meaning from the attackers perspective, he's not risking anything and in fact it's easier for him by spreading out his infrastructure, so all different machines, so if some administrator notices, oh wait I'm serving malware, I should probably not be doing that. It wasn't a horrible blow to these cyber criminals, they're like, OK next server, next!

Dave Bittner: [00:10:20] So they've got a widespread distributed network of compromised machines that they can implement to do their business here.

Daniel Goldberg: [00:10:27] Yeah. We're talking at least 300 attacking IPs just in a one-month period, and we saw a similar number of files servers spread around the world.

Ofri Ziv: [00:10:39] I think that this infrastructure, it makes lots of sense for an attacker because stopping him become much tougher. It is basically very hard for us as a security community to stop these attack at once. So what we can do basically is we can we can try to block it in different places, but overall, the fact that it is using so many servers across continents, across countries, makes the blocking procedure very very complicated. And another thing, and this is maybe something that you can look at to distinguish between maybe criminals which might be more advanced, because they are using these distributed infrastructure. Those guys are more advanced than maybe like a script kiddie or someone that just tried to run something from one single host that he owns. I think the second step that we didn't see here, and this is maybe what will distinguish those guys from the larger APTs and maybe nation cyber operations, is the fact that eventually those guys are attacking from compromised machines, but they are uploading all their capabilities on them. And I think that the next step what maybe makes the larger APT operations, is that they are attacking through a compromised machines but they will never host their, I don't know, exploits, and there are tools on them, because if one of those machine will get hacked back or if someone would try to look deeper into it, they wouldn't want their tools to be captured by someone else.

Dave Bittner: [00:12:21] So, Ofri, in terms of prevention and mitigation, what are your recommendations there?

Ofri Ziv: [00:12:26] So I think that there are several different things that need to be taken care of. First of all, people should as always need to make sure that they are using strong credentials. If they can use two factor authentication, those are like the obvious things. But as we saw in the past and we keep seeing it over and over again people have hard time to be able to actually manage their credentials properly and even their servers that are exposed to the Internet. So I think that in order to actually be able to prevent such incidents to happen in your network, what you need to do, is you need to be able to actually be aware of all your internet facing services, is a first thing. Many of those services that we saw that got compromised in this campaign are servers that I would say people, I'm not sure that people are actually aware of them. Some of them weren't patched for a long time, and some of them were with very very, their credentials, was I know they used default passwords or other and un-methodological, methodological from the defensive side. Being on top of what you have, what is facing the Internet, is a very very important first stage. And then when once you know this, it will be much easier for the IT manager or the security officer or such a network to make sure that those servers are patched all the time, that they are using the most advanced credential methods. I think those are like the two key things as we've seen in such a company like the Hex-men.

Dave Bittner: [00:14:12] And Daniel, in terms of attribution, who do you think's responsible here?

Daniel Goldberg: [00:14:16] So attribution is always something scary to do because we don't have evident proof, but in this case it's really clear that we can tie this to Chinese and cyber criminals as the same thing. In this case, we're talking about dozens of examples of Chinese comments, Chinese e-mails, using different code fragments focusing on Chinese software. They have an example where they mimic a very popular Chinese music streaming program as one of their Trojans. We don't know the name of the developer, know where he lives, but we definitely know that his email address is a popular Chinese service. He writes comments to himself and Chinese, the compile paths are Chinese. None of this is a smoking gun, but it really adds up really quickly.

Daniel Goldberg: [00:15:06] And the second thing is, this is a very much criminal-oriented enterprise. We're seeing both cryptocurrency mining, there's a DDoS component, it's not really about, let's stick around here for the next two years extracting information. It's more, in the trend of, OK we we got in, let's get the maximum value we can out of this host, while they leave themselves the option to exfiltrate data, or ransom data and stuff, but they're really focused on let's get profit running out of this machine.

Dave Bittner: [00:15:41] It was interesting to me in your research, that you described how they'll get in and use a machine for a certain amount of time, and then they'll get out and move on to another machine.

Daniel Goldberg: [00:15:50] Yeah, so this is very much depends on the variant, but they don't, for example, as part of their compromised infrastructure they don't stick with the same attacking server or scanning server for more than a few days, or max, a week or two. We don't know their internal thought process but it could be like, OK, we have something, we got what we wanted before anybody gets suspicious.

Dave Bittner: [00:16:14] So explain to me, to sort of take a step back at a high level here, what is, why are these three grouped together, Hex Taylor, and Hanako? What's the common thread between them?

Daniel Goldberg: [00:16:26] They have multiple common threads, in a way. One of the main things, they share attack techniques. Up until a very late stage, they run the exact same commands which is not very likely for an independent attacker to do. This isn't some exploit kit you download from the Internet, this is something like they wrote, they debugged, they use and we see them iterate over it. We also see a lot of shared infrastructure, which is pretty much the smoking gun. The same IP can be used to attack and deploy both a Hex malware binary and a Taylor binary in the next attack depending on the time, the time of the attack, they have the same scripts and are sending money to the same cryptocurrency wallets. For all intents and purposes, we can't tell the different attackers apart. Maybe there's some sub-group that's building this botnet and that botnet, where we we sometimes see that, but this case it's they're really working together.

Dave Bittner: [00:17:24] I see.

Ofri Ziv: [00:17:24] Yeah, and the thing, and another thing that we can add maybe about those attackers, is the fact that they are evolving. So I mean we've been tracking them for several weeks and maybe a bit longer, and we saw how those attacks are actually becoming more and more sophisticated. Those guys that they learn. The learn fast and they make their tools better. They add more mechanisms to it not being caught by different security products, that are good and they they're getting even better and I'm sure that once they will get over with those three variants they will continue to do something else, something even more advanced. So I think that attackers just as the defenders, they are getting better and better. Very interesting things are ahead of us.

Daniel Goldberg: [00:18:11] A lot of hype, for obvious reasons, go to the flashy and sophisticated attacks. In this case, this is a botnet of, the very minimum high thousands of machines, all of them servers. And it's alive and is going to stick around because, as Ofri said, it's going to be very hard to take down. And this is going to cause a lot of real damage. We're talking thousands of database servers. Let's be honest here, which are probably containing customer data, patient data. We don't know what's going on behind there, and that was compromised. And we see this happening again and again, meaning we can't just focus on the big flashy stuff. We can't just focus on the latest Meltdown or the latest zero-day, when this is the second botnet and we found just this year, that's taking tens of thousands of machines for its own purposes. At some point, we need to focus back on the basic stuff. And a second is, again we're seeing again and again, patch yourself, handle yourself and these guys are getting in by old vulnerabilities or brute forcing passwords. To use the really cliche statement, it's 2018, really?

Ofri Ziv: [00:19:24] Yeah, and I think that maybe this specific, what they are using, in the end, maybe not like the most dangerous thing for us, or even their victims, because, OK so maybe they will need to pay more money on the power that their machines are taking, or they might be, their network might be used by someone else. I think the more danger part is what will happen next. So, what we saw is, we saw DDoS, we saw cryptocurrency, but we also saw keylogger and the backdoor. So, these variant can, at a later stage, I know this machine can be sold to someone else, or be used by these attackers for another purpose. They might take these, ... maybe this is a single server that someone forgot to patch or or didn't notice about. And from the point, those attackers can move forward inside this network. As Daniel mentioned before, we keep seeing people making mistakes. You know everyone makes mistakes and it's natural that I don't know any network that cannot be breached somehow, from some, using some attack vector. And I think that it is very important to also not only invest in the perimeter and on servers that are actually exposed to the internet, it is also very important to think about what will happen next. So what will happen once one of my internet-facing servers will get hacked? Will my defense be able to detect other lateral movements inside my network? What will the attacker be able to fetch from this point? And I think this is another very important lesson that we should take from this example.

Dave Bittner: [00:21:12] Our thanks to Daniel Goldberg and Ofri Ziv, from GuardiCore Labs for joining us. Their full report "Beware the Hex-men" is available on the GuardiCore web site. Thanks to the Hewlett Foundation Cyber Initiative for sponsoring our show. You can learn more about them at The CyberWire's Research Saturday is proudly produced in Maryland out of the startup studios of DataTribe, where they're co-building the next generation of cyber security teams and technology. It's produced by Pratt Street Media. The coordinating producer is Jennifer Eiben, editor is John Petrik, technical editor is Chris Russell, executive editor is Peter Kilpe, and I'm Dave Bittner. Thanks for listening.