APTs transitioning to the cloud.

Dave Bittner: Hello everyone and welcome to CyberWire-X, a series of specials where we highlight important security topics affecting organizations worldwide. I'm Dave Bittner. Today's episode is titled, APTs Transitioning to the Cloud. Cloud attacks have become so widespread that the Department of Homeland Security has warned against an increase of nation states, criminal groups and hacktivists targeting cloud-based enterprise resources. APTs such as Pacha Group, Rocke Group and TeamTNT have been rapidly modifying their existing tools to target Linux servers in the cloud. Modifying their existing code to create new malware variants which are easily bypassing traditional security solutions. How do we address this problem? In order to detect and respond to these attacks, security teams need visibility into what code is running on their systems. A program note, each CyberWire-X special features two segments and the first part of our show we'll hear from industry experts on the topic at hand and in the second part, we'll hear from our show sponsor for their point of view and speaking of sponsors, here's a word from our sponsor, Intezer.

Dave Bittner: To start off our show, my CyberWire colleague, Rick Howard, speaks with Jonas Walker, security strategist at Fortinet. And later in the program, my conversation with Ell Marquez, Linux and security advocate at our show sponsors, Intezer.

Rick Howard: I'm joined by Jonas Walker, he's the security strategist for Fortinet Labs, welcome to the show Jonas.

Jonas Walker: Hey Rick, thanks for having me.

Rick Howard: You have been advising Fortinet customers for the past 20 months or so about, among other things, threat intelligence and specifically adversary group campaign awareness. So when you talk about adversary group campaigns with your customers, how do you describe them? In other words, what are you trying to convey to them when we have that conversation?

Jonas Walker: The Internet is like the Wild West and there are attacks happening every single second from all over the place. So we need to be prepared against all these attacks, especially in, you mentioned it, the last 20 months with attack surfaces changing all the time .So, it is hard on the defensive side and we need to keep our guard up to date every single day.

Rick Howard: I was looking at the ThaiCERT search page this morning on a completely different manner and they are tracking some 300 adversary groups, so there isn't consensus about the number, but we all agree that there is a finite number out there, would you agree with that?

Jonas Walker: I definitely agree and sometimes, I mean attribution is such a hard game, but I do believe some of these groups, they just have different persona's and they are the same people behind different kind of groups, just to make it more difficult for us. But honestly, in the end, I don't think it matters that much, whether it's 50, 100 or 500, what matters more is that we prepare accordingly against these kind of attacks, which have a very big overlap when we compare these kind of groups. Let's say for example when we talk about phishing attacks, it doesn't really matter which group does it, where this group is coming from, we need to prepare accordingly against these kind of attacks.

Rick Howard: Well I think I disagree with you a little bit there because one of my pet peeves is that the industry, both the vendor community and the practitioners, who do the day to day work, we tend to focus on those technical things, the things that we are sure about, the latest malware, the latest zero day exploit, the latest ransomware. But in my mind, that's the wrong focus, instead of trying to prevent random technical tools from working within my environment - it's a never-ending battle by the way, there's gazillions of these things, so you can never know if you're going to win that one - I believe that we should all shift focus to preventing the success of adversary group campaigns. Instead of blocking one tool with no context, I believe we should be blocking adversary group campaigns at every stage of the intrusion kill chain and I'm wondering what you think about that?

Jonas Walker: Defense in depth and having these kind of different layers, properly secured, I definitely agree on that. I didn't mean to say that it's just single groups which we need to focus on. I definitely agree the kill chain is there for a reason and we have these different kind of layers which we all need to secure properly because there is no 100% security on just one single layer and we need to make it as difficult as possible on every single layer.

Rick Howard: So the intrusion kill chain idea has been around since 2010 or so, most of our listeners are familiar with the concept but for those that need a refresher, can you explain what it is?

Jonas Walker: We have the different steps and when we think about these attackers, they don't just randomly start a terminal and start hacking, they usually come up with a plan and these plans are usually divided into different kinds of status. The first step is usually the reconnaissance and this is also the stage where attackers spend most of their time, sometimes weeks, months, where they prepare targets in advance, they try to gather as much information about their potential victims, is it either on social media, open source intelligence, gathering data breaches from the past, to gather possible credentials. Once they have gathered enough information, they come up with a plan during the weaponization part. So they figure out what kind of weapon do they want to use to strike against these kind of possible victims. For example, is it a phishing attack? Do they need the microfiles? Do they want to leverage technologies like PowerShell or different kinds of solutions?

Jonas Walker: Sooner than later, they need to come up with a delivery, so how do they transport this kind of malicious software for example, to the victim and once it's getting executed on the victim, then it installs on their client itself. They need to have communication established and usually this is done with command and controlled servers, where you have an IP address or a domain name which is reachable from the internet, so no matter what they want to do, they can always be in touch with their victims which they were able to breach and give them further control.

Jonas Walker: Then last but not least, it's about the action on the objectives because these attackers, they have different kind of goals. For some of them it's all about money, for others it's more about espionage or sabotage. So it very much depends on the attacker, what kind of objective does he have? But these steps in the kill chain, more or less are there for every single attack. For some of the attacks they matter a little bit more, for some others a little bit less, depending on what the attackers try to focus on.

Rick Howard: So to piggy back on what you said, and I really like what you said, it isn't just one attack that makes these adversary groups successful, they have to string together a series of steps to be successful on what they're trying to do, whether they're stealing data or destroying it or doing some sort of ransomware operation. If you look at the MITRE ATT&CK framework, you look at adversary groups and how they do things, those steps could be as many as 30 to 300 steps, depending on how complicated and how mature their campaigns are. So what I have been advocating for is we should be looking to prevent each step in that attack sequence and not just worry about the individual tools.

Jonas Walker: I definitely agree and if you just look at the MITRE ATT&CK framework, we see so many different techniques just for example initial access and this pretty much depends on the attacker's strategy. Because during the reconnaissance phase, they probably figured out what is more likely to be a success element in this whole attack. Is it more likely to gain access with phishing or do they find potential websites with outdated vulnerabilities where they can just leverage some exploit, which are known to everyone, and use them to get initial access to these kind of environments.

Rick Howard: So, since the beginning of the kill chain idea, threat intelligence groups from the vendors like Fortinet Labs and from the private sector, we've been tracking these adversary group campaigns across the intrusion kill chain. But my observation is that these campaigns generally go after the victim's traditional infrastructure and not really their cloud assets. They establish a beachhead on an employee laptop or a mobile device or servers in the data center and then they move laterally to find the data they have come to steal or to destroy. They do that by stealing credentials and then finding a way to elevate those to some privileged account somewhere. When the data they're looking for is in the cloud, they just use these stolen credentials to get access. But they install their command and control system in the traditional on-prem infrastructure and not in the cloud and I'm curious about whether or not that pattern is changing. Have you seen any examples of adversary attack campaigns using cloud infrastructure across the intrusion kill chain?

Jonas Walker: We do see both, but I do agree with you, the vast majority is what you mentioned with the traditional attacks and I think the reason for that is that if you do have credentials, you pretty much don't need to hack any technologies or any systems, you can just log in because you have a proper user name and a proper password. But one example which we have seen just recently with cloud attacks is that so many enterprises put their public and private clouds directly to the internet. Let's think about all these private cloud solutions which I'm running as well by myself in my old data center with either vCenter, Azure or OpenStack. If these management systems are directly connected to the internet and vulnerabilities are disclosed from these vendors and they recommend patching as fast as possible. We see immediately very high spikes in traffic, attacking these vulnerabilities and a lot of these proof of concepts of how to exploit these vulnerabilities, spread very quickly across the internet.

Jonas Walker: If attackers are able to use these exploits which in some cases are remote code execution exploits, which means all they need is internet access and find one of these vulnerable management systems directly connected to the internet, then they can log in without user credentials and they can access all the vehicle machines, they can find some storage devices, they understand what kind of backups are connected to these management systems and maybe delete them before deploying the ransomware, so they are more successful with their current attack campaigns.

Rick Howard: So you talk to Fortinet customers about how to think about protecting those cloud assets, what is some general purpose advice you give them about preventing these attacks on their cloud infrastructure?

Jonas Walker: So, for example, when it comes to the sophistication of these attacks, I think more often than not, these attacks are not so super sophisticated. Unfortunately very often, the initial access gets achieved by poor configuration from the defender side, shadow IT, which people are not aware of, misconfiguration or just people being under business pressure. What they do is they put their development environments in the cloud, sometimes forget about it. What I have seen recently happening with the pandemic is that people were changing their environments very quickly due to certain lockdowns and due to working from home and it was all about availability. But as we all know, security is not just availability, there's very important factors like confidentiality, integrity and if we just think about availability and hope we can put in security layers at later stages and just make sure we can connect very quickly, then whoever has internet as well, uses different kinds of search engines which scan all these new systems on the internet and if they have bad configuration management or vulnerabilities which are out of date, then it's not that difficult for these attackers to gain access to the systems.

Jonas Walker: So, in my opinion, security needs to be top of mind and it needs to be priority number one when it comes to putting stuff on the internet. Because if we don't, however has internet can do pretty much the same as the ones with the credentials.

Rick Howard: So, I agree with you that most of the cloud hacks we've seen in the last five years or so, even the last ten years has been leveraging misconfigurations leaving the doors and windows open, so to speak. But I'm wondering what you think, why that is, you know, AWS started in 2006, so we are well past a decade of using cloud infrastructure for our organizations. You'd think we would have got better by now, why are we still messing with configuration problems and not really worrying about stopping adversaries. What do you think the problem is there?

Jonas Walker: It's a good question. I think one big reason is usually rushing time lines, this is one thing which I see very often that the business just demands things to be in place at a certain day, so we have this pressure. Also, sometimes, as we know, people are sometimes unfortunately the weakest link and maybe they are not properly trained, they are overworked. We have a lack of very skilled people in IT in general, so it's a hard battle to fight. And also the cloud is quite complex, so if we don't know exactly what we are doing and information gets leaked in the wrong way, then this can be abused for attackers very dramatically.

Rick Howard: Well, I like the idea that you said it was complex and it isn't like we've replaced cloud complexity or the old infrastructure complexity with cloud complexity, we've added it because we're still operating in our old data centers back on prem, and we've even added mobile devices too. Plus SaaS applications and now a lot of us have a lot of workloads inside of hybrid cloud environments, so the complexity of the security environment has really sky rocketed and maybe that's the reason that we can't even get the configuration done right, what do you think about that?

Jonas Walker: Yes, definitely and also people are not just connecting from their secure office spaces, people as you mentioned, are using mobile phones, they're working from home, they're traveling, they're at different locations and the complexity is definitely something which makes it more difficult and if it's not properly configured then this is a really big deal.

Rick Howard: So Jonas, let's end it with this, can you offer any advice to the listeners today about what should they be thinking about first in securing these cloud environments? What's the number one thing that should be on their minds?

Jonas Walker: I think we need to keep in mind that whatever we put on the internet it doesn't take a lot of time until someone tries to hack in. So, we need to keep in mind that before we put something on the internet, we come up with a plan, we have a strategy, we have policies, procedures in place, how to properly secure these environments and for that we need to understand what are we actually trying to achieve? And security in the first place sounds like a lot of additional work and makes everything a little bit slower, but the drawback is, if we don't do these things and put these devices directly on the internet, I see these scanners out there picking up these new IP addresses in the first couple of hours and then getting automatically scanned against known vulnerabilities and being under attack immediately. So keeping in mind that whatever we put on the internet will be attacked very quickly and if it's not properly secured, it's pretty much game over.

Dave Bittner: That's the CyberWire's Rick Howard speaking with Jonas Walker from Fortinet. Next up, my conversation with Ell Marquez, Linux and security advocate at Intezer, our show sponsors.

Ell Marquez: We all talk about that transition to the cloud, it's a topic that is overly discussed but we like to focus on the positives and bypass the negatives. And to me, it really is the emphasis that we have placed on agility. We want to be the first to market with new features, we want to be the first that integrates new technology and what we're doing is, we're putting it all on our devs. Alright devs, you're in charge of deployments, you're in charge of security and your own workload, in charge of securing the cloud, in charge of patching vulnerabilities. Then we look over at the security department and go, "Alright, make sure nothing gets messed up." It's impossible, right? We provide all this defense in depth, yet we tell security teams, "We'll get to training, right now we need to get this feature out, right now we need to do this," and we never get to the training part, we never get to actually teaching us on how to secure our clouds.

Dave Bittner: Can you help us understand what are some of the benefits and liabilities when it comes to working in the cloud? Specifically when it comes to this stuff, is this a case where there are benefits but also perilous things as well?

Ell Marquez: It's interesting because I think that the benefits are also the negatives. We have what I talked about with agility and with that comes, things coming in, things going out and we lose visibility into what we're actually protecting. You talk to security teams and they're like, "We have to defend our customer's data." Great, where's that located? Databases, but how many databases do you have out there, especially if they're not on prem? How quickly is this information coming in and out? We also have the fact that many companies are offloading the infrastructure. Infrastructure is the service, don't worry, the cloud provider will handle it. How often do we speak to the cloud provider on their security status? Right now it is supply chain all the things, that's all anyone is talking about. Imagine if an advanced persistent threat was actually able to compromise the cloud provider themselves? Have you even asked your cloud provider if they would let you know that that occurred?

Dave Bittner: Wow. It strikes me that perhaps there's a false of sense of security when it comes to moving things to the cloud. Do you think that's accurate, that people think that the big name cloud provider is going to maybe provide more than in reality they actually do?

Ell Marquez: I have heard that so many times and it's scary to hear that it's still happening where people and companies say, "The cloud is secure by default." The whole concept of the shared responsibility model where I just offload that security. We're using new things such as functions, but we just assume there's security and Intezer has proven that that's not the case when we were able to hack Azure Functions. These are extremely difficult times because we want to believe in our cloud providers and we end up in a catch 22, do we focus on security there and ensuring it's there or do we focus on security on our end? It's just too broad, there's too much.

Dave Bittner: Let's talk about some of the APTs themselves and some of the things that you and your colleagues are tracking when it comes to how they're exploiting the cloud environments.

Ell Marquez: Recently in 2020, we actually saw an increase of 40% in new Linux malware families and many of these are coming from APTs, for example, AP28 and AP20 and you can just add all the numbers here. What they're seeing is the vulnerabilities that we've introduced into our system. We're so busy configuring our cloud and ensuring our cloud deployments, that we leave things like Docker APIs open and we've seen and discovered malware such as Doki, that's specifically taking advantage of this. In the news currently is TeamTNT that's actively going through and finding cloud resources and getting legitimate credentials, sitting and watching the servers, to see what our workflow is, so they can bypass that signature in anomaly-based detection. It's kind of like a prime target for them because they have the time to actually learn and invest, whereas I've mentioned several times, our security teams lack that time. So we have APTs that have greater visibility into our systems and we've even seen them come in and actually do the hardening on our systems to make sure that they're the only ones that are in it. They're better at our security than we are.

Dave Bittner: I've seen some of those stories where bad actors will come in and will actually clear out other old group's malware, so they have exclusive run of the place, that's fascinating that they're putting in that amount of effort.

Ell Marquez: The most fascinating story that I've heard around there is, I would say, a turf war behind, and I'm going to mess up these names, Pacha, Rocke Group, hopefully I got those somewhere right. What we're seeing is them actively going after one another's crypto jacking software and keeping each other out of the servers, kicking each other out, hardening. All the threat actors seem to what the same thing, they want the resources on our system, so eventually they're going to be finding the same vulnerabilities. It's just a fascinating thing to me that they've, along with working with each other, have turned on each other.

Dave Bittner: No honor among thieves I suppose?

Ell Marquez: I think historically we've seen that happen over and over, so why not the cyber security field?

Dave Bittner: Now is it accurate to say as well, as you eluded to earlier, is that the easy, global availability of these cloud service platforms means the bad guys have that same level of availability. What makes it attractive to us, which is that we can use it easily from anywhere, means that they can have that same sort of access that we can.

Ell Marquez: They really can and we constantly see them using especially cloud providers outside of the US, outside of countries where they have that accountability to say, "This is who used it," and at times they even use infrastructure from countries that do have that protection because of all of the red tape involved within it. They can be gone by the time we're able to get to the actual investigation.

Dave Bittner: What is to be done and what are the potential solutions and recommendations for how folks can best prepare themselves and protect themselves against these sort of things?

Ell Marquez: We do need to continue doing some of the traditional things, we do need that in point protection. Obviously, we need to ensure that we have visibility into everything, from our in points to our servers, but at the same time we need to ensure that we actually know what's running on our assets. If they're coming up and going down with a matter of seconds, do you have something like Doki that would have bypassed your early screening because the image itself was not malicious. Then it was able to use a command which many servers have, kind of a living off the land attack, by curling down the malicious payload. That happened in a matter of seconds to minutes, were you able to detect that at the time?

Ell Marquez: So, by monitoring the runtime on our assets, we can actually get a view onto the code that is happening within that time. And one thing that we're noticing, especially when trying to look for APTs, is that there's a lot of shared code, they know these attacks work, why would they reinvent them? They just start adding new code, changing a bit of things in order to bypass that signature detection. They share code with one another, we basically have malware as a service at this point and they start adapting features a little bit from A, a little bit from B, in order to gain different capabilities.

Dave Bittner: We mentioned at the outset that one of the really attractive things about cloud environments is how nimble they are and how quickly you can do things. Does adding this sort of thing, having a system in place to keep an eye on what code is running, does that take away from an organization's ability to be nimble? Is that an extra load to lift or are they able to pretty much run in parallel without a real impact?

Ell Marquez: At Intezer, one of the things that we focused on is ensuring that obviously we don't add to the resources that are being used within a system. If you've been compromised and you have crypto jacking occurring, then you're already at high CPU levels and by focusing on sensors and technology that monitor things as they're occurring, that only really alert and notify you when they see malicious code, when they see malicious activity being launched, it takes a big load off what your security teams are having to do. It takes a load off the resources that you are dedicating in order to monitor and protect your systems. One of the things that I heard recently is that cloud servers and cloud resources are being attacked every 90 minutes, and that's quickly going down in time, not going down in frequency, but the time used. It's extremely complicated and overwhelming when we look at our security teams trying to maintain our up-time and the correct resources to assist them. There has to be a balance between it and that's why I personally, and Intezer as a company, believe that that run time protection is so critical.

Dave Bittner: What are you hearing from the folks out there who are implementing these types of systems, the people who are charged with defending them? When they have something like this in place, how are they able to do their work, their lifestyle in general?

Ell Marquez: I hate to say that the thing I found most entertaining, almost uplifting, was that they're saying that they're gaining time to be able to do more of the analysis, more of the DFIR things. When we're so quickly trying to put bandages on what's going on, many things slip through the cracks or we just harden our systems and try to move on. But with this extra time they're able to delve in, have greater root cause analysis. In fact, many of them are surprised at how many libraries and binaries are actually shared between all of this malware, which makes it so much easier for their teams to be able not only to detect it, but also to be able to build stronger defenses against it.

Ell Marquez: One of the things that we definitely want to focus on and it's something that I talk a lot about and government agencies talk a lot about, is that we need to have that assume breach mentality. Yes, we can build all of the defenses we want, but when we're so busy building these walls, we're kind of building an environment where we're helping the APT that is already in our system. We need to be able to focus on our actual systems and look for the attacker that's probably already in there. I did a lunch and learn where I spoke to a lot of companies and 90% of the people there, along with saying that they didn't have visibility to their assets, said that there is more than likely something already compromised in the environment that they just don't know about. So, critical with this, is assuming breach and looking to see what's actually occurring on that system. Because the attacker, more than likely, is already there.

Dave Bittner: On behalf of my colleague, Rick Howard, our thanks to Jonas Walker for sharing his expertise and to Ell Marquez for joining us. CyberWire-X is a production of The CyberWire and is proudly produced in Maryland at the start up studios of DataTribe, where they're co-building the next generation of cyber security startups and technologies. Our senior producer is Jennifer Eiben, our executive editor is Peter Kilpe. I'm Dave Bittner, thanks for listening.