ICS honeypots attract sophisticated snoops.

Dave Bittner: [00:00:03] Hello everyone, and welcome to the CyberWire's Research Saturday presented by the Hewlett Foundation's Cyber Initiative. I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down threats and vulnerabilities, and solving some of the hard problems of protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us.

Dave Bittner: [00:00:26] And now, a moment to tell you about our sponsor, the Hewlett Foundation's Cyber Initiative. While government and industry focus on the latest cyber threats, we still need more institutions and individuals who take a longer view. They're the people who are helping to create the norms and policies that will keep us all safe in cyberspace. The Cyber Initiative supports a cyber policy field that offers thoughtful solutions to complex challenges for the benefit of societies around the world. Learn more at hewlett.org/cyber.

Dave Bittner: [00:01:02] And thanks also to our sponsor, Enveil, whose revolutionary ZeroReveal solution closes the last gap in data security: protecting data in use. It's the industry's first and only scalable commercial solution enabling data to remain encrypted throughout the entire processing lifecycle. Imagine being able to analyze, search, and perform calculations on sensitive data, all without ever decrypting anything. All without the risks of theft or inadvertent exposure. What was once only theoretical is now possible with Enveil. Learn more at enveil.com.

Ross Rustici: [00:01:42] So one of the things that we're really curious about is modeling out what isn't necessarily reported often by the security industry writ large.

Dave Bittner: [00:01:51] That's Ross Rustici. He's a Senior Director of Intelligence Services at Cybereason. Today we're discussing research he coauthored with Israel Barak. It's titled "ICS Threat Broadens: Nation-State Hackers Are No Longer the Only Game in Town."

Ross Rustici: [00:02:07] We know there's a lot of tactics and techniques floating around out there. We know that certain types of activity tends to garner more press and more coverage than others. And so we decided to do kind of deep dives into various specific circumstances to see if there is a wide variance between what's the top tier of what you read about from a threat-profiling perspective, and what these institutions and these specific sectors are really facing. And so we've been doing honeypots, one about every three to four months now for about the past year, to kind of find that delta and see what interesting things are going under-reported because everybody's focused on the cool, new vulnerabilities and interesting new pieces of malware that are out there.

Dave Bittner: [00:02:55] So, before we dig in, can you describe to us what exactly is a honeypot?

Ross Rustici: [00:03:00] Yes, so a honeypot is essentially a fake computer set up to be exploited. Most honeypots are really simple setups - it's an SSH server, it's something that is fairly low-profile, fairly low-functioning - but they're set up in such a way that they have weak protections, so that way they get exploited and a security researcher can see how they're exploited or what a hacker is trying to do by exploiting it.

Ross Rustici: [00:03:30] What we've done is kind of put the honeypot on steroids and done a full honey-network. We've got a completely virtualized environment that has fake traffic flowing through it, and it looks like a small-medium-sized company or a division within a much larger company, which gives us a lot more telemetry and rich detail on what the hacker does because, once they get on a computer, they actually see a live network and so they try to exploit it further, and we get more information both on what second-stage tools they're going to use and also what they're actually after in that particular network. And so, the modeling aspect of it is a higher cost on that front-end for us, but it's paying dividends on the back-end in terms of getting rich telemetry from these hackers.

Dave Bittner: [00:04:18] Yeah, let's dive in. I mean, it's an interesting thing that you all are working on here. Take us through, what did you set up?

Ross Rustici: [00:04:24] This most recent one, we set up a fake network that looks like a very large electric power supply company. And we use some doppelganger domain names, we set up the IP addresses to be very close to the range of a known industrial supply center. And as a result, we got a lot of activity. Within 24 hours of going live, of spinning up this fake network, we were seeing lots of activity going against it. Almost immediately, we saw a toolkit that's been known with an underground hacking forum penetrate the network and set up fake accounts to then go sell the access.

Ross Rustici: [00:05:06] And then, within about 24 hours of seeing that initial breach and set-up, we saw somebody come in through one of the accounts that were set up and start very methodically moving through the network, attempting to gain access to the industrial control systems that, at least from a network perspective, said they were there. Obviously, this network does not actually control anything, and so there's been some interesting technical challenges of making it look like it's got industrial control systems operating, and that's what these hackers were going after. So, within 72 hours, we had full exploitation of one box, sold access, and a group of hackers coming in and trying to move laterally, as efficiently as possible, towards the industrial control system.

Dave Bittner: [00:05:52] When you set this system up, were you intentionally making it relatively easy to get into, or were you trying to have a degree of security that would be comparable to what an actual provider of electricity would have?

Ross Rustici: [00:06:05] Both. So, some of the web-facing assets had very weak passwords and usernames for the remote desktop protocol service, which is one of the ones that hackers love to compromise because it allows you to be interactive and see what's going on on the desktop as you do it. And then we put in some layered defenses so that way it would be hard to laterally move directly to the ICS system, so that way we would force them through more computers so we would get a better understanding of how they're moving through the environment and what their preferred methods were. So the initial compromise was really easy - getting to the industrial control systems was significantly harder. And I think, given the way networks are generally set up, that's more realistic than it probably should be.

Dave Bittner: [00:06:51] Can you walk us through kind of step-by-step, the process and the tools that they used to get to the things that they want to get at?

Ross Rustici: [00:06:58] Yeah, so, in this case, it's been really interesting because the hacking group that initially compromised the system used a bunch of well-known hacking tools that basically were anti-forensics as a way to sell the access. So they laid down a tool that modified the RDP service in such a way that two people can be logged in via RDP at the same time and it won't boot anybody off. So, that was their way of assuring access that they're about to go sell.

Ross Rustici: [00:07:31] The more interesting activity, though, was the guys who came in afterwards. They've been living off the land almost entirely. They've been using PowerShell scripts. They've been using local admin commands - netstat, those types of things - during their internal recon, and we haven't actually seen them pull down a single tool, which shows that they have a higher degree of capability than your average dark market script kiddie who's just looking to explore something. They know what they're going after and they're using only internal resources to get there, which means they're trying to keep a very low profile.

Ross Rustici: [00:08:11] The other really interesting thing that we saw almost immediately was, when they landed on the box, they started uninstalling the security tool. Obviously, we're an endpoint detection company, and so we laid down our own security tool there, unhardened, just to see what they would do. And within about a half hour of landing on the box, they uninstalled our probe. So, we spun it back up and hardened it slightly to see if it was just kind of matter of course that they do this, or if they were really concerned about being caught. And we left one path open for them to uninstall the probe again, and within two hours they uninstalled the probe again.

Ross Rustici: [00:08:48] And that kind of aggressive anti-monitoring capabilities isn't normally associated with your run-of-the-mill hacker. We then reinstalled the probe with full hardening, and it's still operational. But the fact that they're going back and forth with us, even just on your reimaging-type, basic network hygiene stuff when you notice compromise, shows a level of full sophistication or brazenness that you don't normally see in these types of hacks.

Dave Bittner: [00:09:16] Yeah, I think it shows a level of tenacity there, because you would think it would be an indicator to them that perhaps someone was onto them.

Ross Rustici: [00:09:24] Yeah, exactly. And the fact that they're willing to go toe-to-toe with, presumably, a level one or level two SOC analyst based off the actions we were taking. We didn't want to spook them out of the network entirely, but we wanted to force their hand a bit to see how they adapted to being caught in some part of the kill chain. They were obviously not concerned with what we were doing and they kept on doing lateral movement while they were going back and forth with us on this one particular box. Which also demonstrates that they probably had at least two people in the environment, because we saw them doing activity while we were doing reimaging and that sort of thing, and they were kind of trying to counter those actions.

Dave Bittner: [00:10:09] So, what was your sense for what they were after? Are they looking to exfiltrate data? Are they looking to get control over some of the industrial controls in the system? What's your take there?

Ross Rustici: [00:10:21] So, they're definitely going towards the industrial control systems in the network. Every time they would hop onto a new box they would scan specifically for the boxes that were identified as running the industrial control systems. What their endgame was once they gained access to those particular machines we still don't know, because they haven't gained access to those machines yet. The environment's still live. It'll probably be spun down sometime Saturday, and we'll do our final triage of data then.

Ross Rustici: [00:10:49] But right now, that's the million dollar question for us - are they trying to exfil data just so that way they understand how these machines are operating for potential future use, or are they looking to do something nefarious in the short term because they just want to see what kind of damage they can cause? We don't have a good bead on that yet. There are a couple more prodding actions that we have planned for later today. We'll get a much clearer understanding of what they're intending to do.

Dave Bittner: [00:11:18] And what is your sense of their capabilities in terms of persistence?

Ross Rustici: [00:11:22] So, they've laid down some persistence in a couple different boxes that they've laterally moved to already. We already took the action of wiping clean the primary landing point that they have, and we haven't seen them try to regain access to that particular box. But we're still seeing activity in the network, which leads us to believe that they are still doing things on the other machines, and they're still using that as their persistence backdoor mechanisms.

Ross Rustici: [00:11:50] That's all been done through scripting as well. They've gone in just to run keys and spun up PowerShell scripts as a result of the way they've laid down things. So, every time those machines get rebooted, a new command shell gets spun up for them. And it's not necessarily very sophisticated in terms of how they did it, but it's very effective if you don't know what you're looking for because you hadn't monitored every step that they've already taken.

Dave Bittner: [00:12:21] And so far there's no sense that they're on to the fact that this is a honeypot?

Ross Rustici: [00:12:26] We haven't seen any indication of it. And the fact that they keep on combating the moves that were taking to try to kick them off leads me to believe they haven't figured it out yet. I have a feeling that, once they actually get to the ICS systems on the network, they'll quickly realize that the whole thing was a charade. And that's going to be an interesting data point in and of itself. Some hackers get really angry when they get caught in honeypots and try to destroy the system. Others back out as quickly as possible. It will be interesting to see if they cut and run or get angry and malicious.

Dave Bittner: [00:12:59] Now, what is your sense in terms of who's behind this?

Ross Rustici: [00:13:03] We don't have enough technical details to broach the attribution conversation with any confidence. What we can say is that the access was gained through a darknet forum. So, it's obviously black hat, somebody who has technical sophistication because they're living off the land. Whether it's one of those guys that swings between cybercrime and nation-state, or just somebody who decided he wanted to go play with a SCADA system and he's relatively technically sophisticated, we don't have enough detail to draw that line.

Ross Rustici: [00:13:38] But I would say this is not your average, run-of-the-mill, script kiddie, and this is not the generic type of stuff that you see in most honeypots. This is very targeted. This is very educated about what they're trying to go after and how to get there. And that is interesting to us, and we're going to try to pull apart the specific attribution data once we've closed down the environment and can do true forensics on it because it's not changing.

Dave Bittner: [00:14:04] As far as the tracking in the dark web forums go, that's something that you were actively tracking and looking for, and what did the - the folks who initially broke in - what were they out there advertising?

Ross Rustici: [00:14:17] So, we did some poking, and it appears that the dark web is not as open as it used to be. This is something we actually found in the last honeypot we did as well, because we actually seeded some access for the old honeypot in the dark web and got zero bites. The amount of trust that it takes to do a transaction in the dark web these days is much higher than it used to be, due to the fact that the FBI has been so effective at shutting down some of the bigger dark web forums.

Ross Rustici: [00:14:49] And so, what we saw with this one is a sale that took place not in public. So, we know that the forum hosted the information, but immediately when it went up for sale it appeared to go into side channels that aren't publicly available unless you're the seller or the buyer. So, we don't have the data on how much it sold for or who the person was who bought it, but we know that it was bought and sold because we saw the initial posting of access for and because the domain of the username and password that was set up by the original person who compromised it was then used for the second stage and more targeted activity. One of the things that we'd like to do is try to contact the original seller and see if we can buy back the access for a higher price, just to see what the original price was.

Dave Bittner: [00:15:48] Right. Right. What broader perspective does this give you all? I mean, if this is the activity you've seen on your honeypot, on your virtual electrical system, what do you suppose this means in terms of what people are able to do with the actual electrical systems throughout the US?

Ross Rustici: [00:16:07] I think there are a couple key takeaways for network defenders, especially in the ICS space. The first is you are not just dealing with nation-state adversaries. I know that that's the primary focus of the conversation these days especially with the DHS release regarding the Russians, and there was a couple other DHS releases earlier this year that dealt with North Koreans and the Iranians.

Ross Rustici: [00:16:31] The fact that this access was bought and so specifically targeted on the dark web shows that there are other people interested in these types of systems that aren't necessarily your upper echelon nation-state actors. That should be a concern for anybody who's running these assets because, at least with the nation-state actors, they're trained on how to operate with these systems - they have the background in it, and they have a specific mission. With the guys who are just interested in playing - the ones that are hanging around the dark net forums - there's a high chance that they're going to make a mistake, and that mistake might actually end up causing a power outage or causing real-world damage.

Ross Rustici: [00:17:11] And that's always been the big concern, from my perspective, with all the targeting of ICS in general. These are fragile systems. They tend to be overloaded as it is. They go down fairly easily. Mistakes are more likely to bring down a power grid than an actual targeted attack. And the more lower-tier threat actors that get into these systems, the more likely that's going to be. So, from a general awareness standpoint, I think this is a really good takeaway that we need to be very concerned not just with nation-state actors but the mid and low-tier actors as well.

Ross Rustici: [00:17:45] The other thing is make sure you have layered defense. Getting into a corporate network is relatively easy; getting to the ICS system should not be. We set it up in that manner to see how quickly an actor can laterally move through a business network to the ICS systems with some level of layered defense. And so far, without taking a really strong hardening approach in the network, we've been able to keep them out of the ICS systems for three or four days at this point. If your network isn't configured that way to begin with, that really should be priority number one if you're managing an ICS system.

Ross Rustici: [00:18:25] Because, at the end of the day, humans are the weak link when it comes to security. And if you have a free flow of information from the business network to the ICS, anybody can move through into it. And they're trying very hard to do so. They can identify the assets relatively easily. And you want to throw up as many barriers and delay their movement as much as possible to avoid allowing them access to the crown jewels of those networks.

Dave Bittner: [00:18:51] I think that's a really interesting insight. The notion that mistakes by the curious, I suppose the maliciously curious, could cause power outages or damage to these systems. Is it surprising to you that we haven't seen more of that, or incidences of that?

Ross Rustici: [00:19:09] I would say yes. I am surprised that we have not seen more instances of lower-tier threat actors making mistakes in critical systems. If you look globally, we've seen some instances with public transportation in Eastern Europe. There's been in some instances with oil and natural gas down in Brazil, if I'm remembering correctly, where hackers have screwed up and caused issues. But they haven't really bubbled up to the point of, this is inexcusable activity.

Ross Rustici: [00:19:48] But I think, as more and more exploits get dropped publicly, as hacking becomes more point-and-click, and less skill is required to do so, we're going to see more and more mistakes. And that's unfortunately just the world that we're living in as the offense continues to grow exponentially due to the amount of data disclosures that are happening.

Dave Bittner: [00:20:14] Our thanks to Ross Rustici from Cybereason for joining us. The name of the research is "ICS Threat Broadens: Nation-State. Hackers Are No Longer the Only Game in Town." We'll have a link to the research in the show notes. You can also find it on the Cybereason website. It's in their blog section.

Dave Bittner: [00:20:34] Thanks to the Hewlett Foundation's Cyber Initiative for sponsoring our show. You can learn more about them at hewlett.org/cyber.

Dave Bittner: [00:20:41] And thanks to Enveil for their sponsorship. You can find out how they're closing the last gap in data security at enveil.com.

Dave Bittner: [00:20:50] The CyberWire Research Saturday is proudly produced in Maryland out of the startup studios of DataTribe, where they're co-building the next generation of cybersecurity teams and technology. The coordinating producer is Jennifer Eiben. Editor is John Petrik. Technical editor is Chris Russell. Executive editor is Peter Kilpe. And I'm Dave Bittner. Thanks for listening.