A Digital Eye on supply-chain-based espionage attacks.

Dave Bittner: Hello everyone. And welcome to the CyberWire's Research Saturday. I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down the threats and vulnerabilities, solving some of the hard problems, and protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us.

Juan Andrés Guerrero-Saade: So this is, as always, great work by Aleksandar Milenkoski. He's in the Labs team. He's always finding some very interesting things all on his own. In this case, he collaborated with Luigi Martire over at Tinexta Cyber. So they were -- they were actually kind enough to bring the initial incident to our attention, and we were able to collaborate on this one.

 

Dave Bittner: That's Juan Andrés Guerrero-Saade, better known as JAGS, from SentinelOne. Today we're discussing their work, Operation Digital Eye, Chinese APT Compromises Critical Digital Infrastructure via Visual Studio Code Tunnels. Well, explain to us what Operation Digital Eye is and why it matters. Who are the primary targets here?

 

Juan Andrés Guerrero-Saade: So Operation Digital Eye is kind of interesting next chapter in this sort of continuum of research that we've been working around this kind of vague APT team, somewhere in the general, like, Chinese cluster. And I say vague APT team because it seems like there is some kind of campaign enablement group or malware development quartermaster that seems to be operating with a variety of other Chinese APTs, primarily to target telcos and certain sort of digital -- digital equivalent of critical infrastructure type targets. I say it's a continuum because, you know, we had originally heard about this as Operation Soft Cell. And then Alex Milenkoski discovered the next iteration, which we called Tainted Love. And now we're at the third chapter with Operation Digital Eye. So our friends are going -- they're still going strong.

 

Dave Bittner: Well, let me just put a pin in Soft Cell and Tainted Love. As a child of the '80s, you'll get me where I live. So congratulations on that.

 

Juan Andrés Guerrero-Saade: I'm glad it resonated.

 

Dave Bittner: Yeah. There you go. Let's talk about some of the methods here. I mean, am I reading the research right, that it starts off with SQL injection for initial access?

 

Juan Andrés Guerrero-Saade: There's SQL injection. And then they -- they use a web shell to try to -- to get their initial foothold in the victim organizations.

 

Dave Bittner: I see. Well, I think one of the key things that catches people's eyes in your research here is the use of Visual Studio Code Remote - Tunnels for command and control. Let's unpack that. For folks who may not be familiar with it, what are the Visual Studio Code Remote - Tunnels?

 

Juan Andrés Guerrero-Saade: So Visual Studio Code is a development environment that's quite common and quite beloved amongst the general software engineering and development community. A lot of folks use this. And it's a really interesting, sort of useful suite of tools for developing code that, since it's so prevalent around enterprise environments and development environments, it tends to get a lot of leeway as far as, like, what's allowed through firewalls, what's allowed on endpoint protection because doing detection for developer machines is actually one of the harder scenarios. They tend to be very unusual kind of endpoints. They tend to install a lot of things. They tend to have entirely different kinds of configurations than your average user, which also means that their tools tend to get a lot of leeway, right. In some cases, folks might even exclude these from getting detected. Our attackers have clearly figured that part out as they took to using the sort of novel technique of taking this Visual Studio Code IDE, this development environment and abusing one of its native features and I think one of the favorite native features, which is the ability to have a remote tunnel to an external system that you use for development. So think of a developer that might have a -- a system in the cloud that's used as part of its CI/CD pipeline or part of her way of developing some of these tools and deploying them into a specific environment. In this case, the attacker saw that capability, saw the reputation of the tool and decided to bring it along themselves. So they're bringing a Microsoft signed executable of VS Code. They're setting it up as a service in the machines that they infect. And then they abuse this Remote - Tunnels capability in order to actually disguise their command and control traffic through the allowances that you would normally make for this.

 

Dave Bittner: And that makes it difficult to detect, yes?

 

Juan Andrés Guerrero-Saade: Extremely, specially on the wire, right. Like, on endpoint level, you know, if you've got a good -- you've got a good endpoint solution, not to shill but you should be able to see some of the behaviors there. But if you're just looking at this on the wire, as far as like the network goes, chances are this is going to get lumped in with other strange but common traffic from Visual Studio Code and from these other sort of development boxes. And since the attackers were angling for that, they went a step further and actually registered their command and control infrastructure on Azure Cloud. And that way, you know, if you're just trying to check your network logs or you're trying to check the reputation of the domains that your environment connects with, well, this is a seemingly innocuous connection from Visual Studio Code to Microsoft-owned cloud infrastructure. What could possibly go wrong?

 

Dave Bittner: Well, before we dig into some of the infrastructure things here, the research mentions that the attackers used custom Mimikatz modifications for Pass-the-Hash attacks. Can you unpack that for us?

 

Juan Andrés Guerrero-Saade: Absolutely. So this is actually where a lot of the connection comes with -- with Soft Cell, with Tainted Love. Alex has done a great job sort of latching onto this set of semi-custom tooling that this digital quartermaster or shared, you know, operations team seems to be using. And part of that tooling is their own sort of modified version of Mimikatz and some of the existing sort of Pass-the-Hash tooling. They've kept modifying it and proving it, changing it to their own liking and, in some cases, even adding some, you know, custom messaging in Chinese for what we assume are other teams that are also working with their tools. So that's a part of our -- that's a part of our, you know, theory around this group, that they are building things that are being used by others. And I think it sort of speaks to a -- perhaps more interesting part of how some of these APT teams like GALLIUM and their attacks of the telecommunications sector and, in this case, the B2B IT sector, how they're going about it, how they are sort of segmenting the work between these different departments.

 

Dave Bittner: I see. Well, you mentioned that they are using Microsoft Azure, and the research mentions European infrastructure for the campaign. How does this help them avoid detection?

 

Juan Andrés Guerrero-Saade: Well, this is something that -- it's actually quite a hot topic these days, as we talk about things like Volt Typhoon, Salt Typhoon, and, you know, everyone's favorite sort of threat actors that are just essentially phasing all of our security mechanisms these days. Part of the new Chinese operational playbook seems to be making sure that the points of exit, the infrastructure that they use to hit their victims is as close to their own, you know, country borders or at least continent in a way that may not arouse suspicion. So, if you think about it, especially as we start to sort of segment the different powers that defenders have across the world, there tends to be -- you know -- you know, think about the US, right? You've got this remit where we have these behemoths like NSA that can do amazing things only from our borders on out. And sort of they're -- they're -- these Chinese APT teams are taking the opposite mentality of saying, well, we're going to make sure that whatever network resources are going to touch our victim enterprise are coming from, you know, as close as possible, as normal as possible so that these defenders are not going to latch on to what's going on.

 

Dave Bittner: The call is coming from inside the house.

 

Juan Andrés Guerrero-Saade: Something like that.

 

Dave Bittner: Yeah. We'll be right back. Can we talk about Soft Cell and Tainted Love? I mean, you mentioned those. Can you kind of explain to us some of the overlaps here that some of the things that for you and your team help connect these campaigns.

 

Juan Andrés Guerrero-Saade: Sure. So Operation Soft Cell was originally reported by researchers at Cybereason, and Alex Milenkoski was particularly interested in this team and actually revealed another attack on the telecommunications sector, along with me and Joey Chen and our friends over at Q Group. So we released that research last year as we kind of latched onto this cluster of semi-custom tooling being used, particularly against telcos, to target telcos. And, at the time, we were sort of putting this in this operational cluster, this unidentified team somewhere in the nexus of GALLIUM or APT 41. And, frankly, you know, if that's gibberish to you, it feels like gibberish to me, frankly, because the whole Chinese APT ecosystem right now is -- it's not just complex. It's actually almost designed in a way that's very difficult for us to cluster properly. So from back then, March 2023, we were already trying to, like, keep up with this group and understand not just who they were targeting or what they wanted from the telcos but also how they related to some of the other teams that they seem to be enabling. So, as we kind of caught on to some of the things that they were doing to customize their tooling, particularly some of these Mimikatz samples, we were able to track some of that development. And Alex recognizes that immediately in this new incident. As you look at some other tooling, it's just a clear evolution from the things that we'd seen modified for Operation Tainted Love. And now, you know, we started to refer to them as MIM Cn. But it's essentially sort of a soft fork of Pass-the-Hash tooling and things that we were familiar enough with but are being improved by this group.

 

Dave Bittner: When you're talking about the Chinese APT ecosystem, and you already mentioned this notion of digital quartermasters and shared vendors, can you help us understand what your perception is of how that works? What I'm hearing is, like, it sounds like things are very fluid. And there's -- there's, as you said, it's hard to pin them down. Can -- can you provide some details there?

 

Juan Andrés Guerrero-Saade: So I think there's two sides to -- to that. The first one, fluid might be a good way to look at it. I would say maybe less regimented. We tend to have this sort of notion of how nation state operations should be run, quote -- you know, quote, unquote should be run. From a Western perspective, we tend to think about authorities and how organizations are divided and how we divide remits and whose responsibility is what. And there tends to be some hard divisions wherein, you know, we've seen in the past, right, it's particularly hard for different governmental organizations to play ball with each other. It seems that the Chinese APT ecosystem or the state-sponsored ecosystem of threat actors has found some way around that. They've found a way to play nice. And what we end up seeing is there's a lot of these teams that are harder to characterize because of some of the tooling that they're using and some of the techniques. But then you also have what appears to be connective tissue between these different groups and clusters of APTs where, in some cases, they're sharing tooling. In some cases, it seems that they might be handing off accesses. Or they might prepare the ground in a certain place and have somebody else come in, some other group come in and kind of finish the job. So it's just a much more complex space. And I -- I'll admit I don't think that this is just coincidental. As you look at the more recent intrusions that are dogging us, particularly in the -- in the United States, there seems to be a certain amount of intentional engineering towards our blind spots, which is what's making things like, you know, the new hot topic du jour of Salt Typhoon such a nightmare for everybody.

 

Dave Bittner: Yeah. Let's talk about detection and mitigation. How were these attacks initially detected and disrupted before they could escalate?

 

Juan Andrés Guerrero-Saade: So there, credit to our friends at Tinexta. So Luigi reached out with knowledge of this new web shell and some of the tooling that they'd originally caught onto, and from there we were able to kind of spider out and rebuild some of the operation and understand how the attackers had moved around, what they had latched onto. And then that's where Alex figures out this VS Code tunneling magic and sort of this new capability. Frankly, it -- as far as detection and mitigation, the advice is getting a lot harder, right? I think we used to come on here and say, Hey. Update your firewalls. Make sure you're checking your logs. Make sure that you are checking the reputation of what network connections happen and so on. It's all very, like, sort of well-rounded advice. But, in this particular case, I think for anybody, you know, any astute readers paying attention to the research, it really wouldn't help you too much to focus too much on the network resources, right? We're talking about this operation being engineered towards that. So we are almost entirely dependent on endpoint protection. And I know that's convenient from somebody -- you know, from someone selling some of the -- these solutions. But as far as from an incident response perspective, it really -- we really don't have many options for protecting these anomalies unless we have great visibility on the endpoints themselves because the network resources are not going to cut it.

 

Dave Bittner: What if I'm somebody who's using Visual Studio Code? I mean, how do I scrutinize a trusted tool like that without turning my normal workflows upside down?

 

Juan Andrés Guerrero-Saade: To be honest, I'm not entirely sure that there is a way for you to do that, right? There have been some improvements to VS Code, in general. And you can see, you know, if you're an avid user, you may have noticed a certain amount of prompting, asking you whether you trust the project that you're opening, whether you trust the code that you're executing. And, I mean, that's all well and good, especially since we've seen, for example, North Korean APT teams targeting developers, targeting exploit researchers with malicious projects. But there's really not an easy way to account for Trojanization and the sort of -- like you said, right; the call's coming from inside the house. In this case, it's very difficult to look at a tool of your own that you love that you're getting from the right place. You're not -- you know, it's digitally signed. Everything is working as intended. And, in this case, it's being turned into essentially a Lull bin, Living Off the Land type of technique. I would go one further when it comes to something like Visual Studio Code and a lot of the tools that developers use. There is a very laissez faire kind of approach to how these tools use plugins. So, for example, VS Code has its own plug-in marketplace. And a lot of it is helpful stuff, and a lot of it is great capabilities, but there is a heavy reliance there on whether you have, you know, good stewardship from Microsoft and whoever else gets to sort of vet that code, that it doesn't become a vector for a supply chain attack. And I say that precisely because you can pull down any kind of plug in that gets put up there. It's going to run in the execution context of VS Code inside of your developer boxes. And, if that sounds like it would be a lot of effort, I would suggest considering the payoff of getting on an engineer or developer's box right. Like, that's -- that's -- that's a key get when you can then turn that into a downstream supply chain compromise. So it's a lot to consider. I don't know what to tell folks when it comes to how to develop policies around these things because it's just very hard to adapt to what developers need. But it's a situation where, if you don't have a good sort of behavioral analytics as far as what's happening with this code in flight once it's running, not when it's on disk, you're very likely to miss the entire thing.

 

Dave Bittner: What are some of the biggest takeaways for you here? When we're looking at Operation Digital Eye, what do you hope folks take away from your research?

 

Juan Andrés Guerrero-Saade: Well, there's a variety of things we could take away. I would actually love to emphasize not just the technique itself and sort of this -- this nifty little novel type of attack but, rather, the level of sustained interest that we're seeing towards -- towards the telecommunications sector, towards the B2B IT sector, MSSPs, other companies that essentially are infrastructure supporters, there is a sustained effort with specific Chinese APT teams that are primarily interested in being there. And that's for good reason, right? Like, it enables all kinds of attacks, further downstream compromises, and general surveillance that is very hard for any of us to defend from, right? Like, we can't possibly look over the shoulders of our own telecommunications providers. We just pay them and hope that they're protecting us. So it's a very difficult situation and one that I think needs a lot more attention from the public at large because the cloud services that we rely on, the telcos that we rely on, they're being targeted quite heavily. And without much of an assurance of their integrity, I don't know that we're in a good position to protect ourselves.

 

Dave Bittner: Our thanks to JAGS from SentinelOne for joining us. The research is titled Operation Digital Eye, Chinese APT Compromises Critical Digital Infrastructure via Visual Studio Code Tunnels. We'll have a link in the show notes. And that is Research Saturday, brought to you by N2K CyberWire. We'd love to know what you think of this podcast. Your feedback ensures we deliver the insights that keep you a step ahead in the rapidly changing world of cybersecurity. If you like our show, please share a rating and review in your favorite podcast app. Please also fill out the survey in the show notes or send an email to CyberWire@n2k.com. This episode was produced by Liz Stokes. We're mixed by Elliott Peltzman and Tré Hester. Our executive producer is Jennifer Eiben. Our executive editor is Brandon Karpf. Simone Petrella is our president. Peter Kilpe is our publisher. And I'm Dave Bittner. Thanks for listening. We'll see you back here next time.