The State of OT Security Part 1: How to Stay Safe in World where OT and IT are seamlessly integrated

Michela Menting: 75% of industrial operators experience a cyber attack in their OT environments. We really need to dispel the myth that such attacks are rare in OT. You know, on the contrary they're terribly common just like they are in IT. And for sure the research shows that the majority of operators have experienced an attack on their OT, and this on a monthly basis.

David Moulton: Welcome to "Threat Vector," the Palo Alto podcast where we discuss pressing cybersecurity threats and resilience and uncover insights into the latest industry trends. I'm your host David Moulton, director of thought leadership. [ Music ] Today I'm talking to Qiang Huang, VP of product management at Palo Alto Networks, and Michela Menting, senior research director at ABI Research. In this episode we're going to explore the critical topic of OT security or operational technology. Michela Menting, Qiang Huang, welcome to threat vector. Excited to have you here today.

Michela Menting: Thanks, David. It's my pleasure to be here.

Qiang Huang: Thanks, David. It's nice to be here.

David Moulton: Qiang, can you provide a high level overview of the most prevalent OT threats we're seeing from our vantage point here at Palo Alto Networks?

Qiang Huang: Sure, David. Before we talk about OT threat I think it's important for us to understand what's going on in these OT industries across manufacturing, energy, and utility. On the business side we're seeing a profound digital transformation that's striving for the mental change of the underlying network. This could be industry zero, smart manufacturing, remote operations, adoption of 5G, and migration. So all of these are really dramatically increased surface in the OT environment. So these are legacy vulnerable OT assets that used to be air-gapped now are getting increasingly exposed to IT and cloud. So with that no wonder we're seeing a lot more OT threats in the industrial organizations. So some of them are even becoming worse in terms of, you know, shutting down OT operations. In our recent survey to about 18,000 industrial organizations we see about almost 70% of them have experienced 1 cyber attack in the last year and unfortunately 1 out of 4 experienced a shut down of the operation. So based on our understanding and the survey the top threats can be ransomware continuing to be a top concern and top threat in the OT environment. Unsanctioned remote connection is another one. Also in different parts of the world -- I just came back from Europe. You know nation state attack is also a big concern.

David Moulton: Michela, Qiang just walked us through some of the things that we're seeing here at Palo Alto Networks. I know you've got a different perspective from your vantage point as a researcher. Are you seeing other areas of high concern, other things that you would want to add in to some of those most prevalent OT threats?

Michela Menting: Yeah. Yeah. I mean what we're seeing is a kind of history repeating itself a little bit. Right? We had, you know, when IT environments started coming online, started being connected, we're in that same phase now with OT, but just 20 years later. Right? We've been covering critical infrastructure and kind of industrial systems for a number of years, for a good decade at least. And those patterns are becoming much more common now, much more prevalent. You know unfortunately attacks against OT are frequent. Right? And Qiang mentioned that, you know, 75% of industrial operators experience a cyber attack in their OT environments. We really need to dispel the myth that such attacks are rare in OT. You know on the contrary they're terribly common just like they are in IT. And for sure the research shows that the majority of operators have experienced an attack on their OT and this on a monthly basis. Right? So it's become recurring and you know this repeated engagement what it means is that threat actors are they're able to achieve a frightening level of success, you know, in their attacks. And Qiang mentioned that, you know, 1 in 4 operators had to shut down their operations last year. So that alone should put everyone on edge. Right? It's a very high level warning sign that something's not going right. And that we're not doing enough from a security perspective.

David Moulton: Michela, if most of the audience isn't aware of what a OT attack or an impact on an operation, an OT operation, means, can you dig into that? What are some of the real world effects of these attacks?

Michela Menting: They can be pretty damaging. We're looking at increased connectivity. We're looking at digitization. We talk a lot about industry 4.0 and we have been for some time. And this means there's a lot of new threat vectors, a lot of additional threat vectors in the OT. You have industrial control systems like PLCs that can have a direct connection to the internet. Right? Obviously you want to be putting that behind a VPN. You want to be putting in place firewall capabilities. You want to create separate networks. You want to separate certain OT networks from others. You know, there should be various layers there before you get to an actual internet connection. Right? So that requires a lot of configuration. And unfortunately that kind of leaves a lot of space for misconfiguration as well. Right? And this is something that threat actors exploit all the time. And even though we think about segmentation and we try and deploy segmentation in OT networks and we have this different -- these number of layers that come between those OT assets and IT networks and in the broader internet it doesn't really stop threat actors. It makes it more difficult, but what we see is despite this layering they continue to be successful. Right? In attacking OT assets. And in part this is because, you know, OT assets, you know, you think about PLCs or programmable logic controllers. I mean you don't patch them as regularly as you would an IT asset. Right? And you certainly don't do that, you know, at production sites. You know you don't want to be disrupting the visibility and control of a plant site or an operation. So you schedule these updates and you might have to do them quarterly. It's certainly not a daily occurrence. Right? And so of course that means for a period of time you have vulnerable industrial control systems, vulnerable OT assets. And even if they -- you know, threat actors come through the IT or originate from the IT space, it gives them time to kind of look around and find those assets that they can exploit. Right? And the research shows that 75% of those OT attacks originate in the IT space. And so what we're seeing is a really sharp increase in attacks against OT because, you know, a lot of different doors are kind of opened and sometimes left unlocked. And so threat actors target these kind of these open doors and things like routine system updates. You know, adding in malicious code.

David Moulton: If I play that back it sounds like you've got a complex environment that once it's compromised puts the operator or the organization into a leveraged position. That leverage tends to lead to ransomware and/or extortion. And that because of the exposure on the IT side you've got now this type of technology that wasn't intended to be connected to the internet and it's suddenly connected to the internet. And I guess where I'm going, and maybe Qiang I throw this to you, is there a high profile OT attack that's occurred that comes to mind? Maybe people will go like, "Oh. That's what they're talking about." And can you talk about some of the impacts to the industry once that occurred?

Qiang Huang: Yeah. Sure, Dave. Yeah. We continue to see high profile breaches and attacks in these industrial organizations. A recent example for example the meat manufacturer like JBS, auto maker like Nissan, we continue to see attacks. What's interesting is that because these OTSS are mission critical, they're revenue generating, so the impact of the breach is also quite high. I think in manufacturing it's among the top three in terms of the impact of a breach into multi million dollar. There's also recent example or Clorox. You saw from the news it's about 20% decline in net in sales in one quarter which is about 300 million. So some of the examples I want to -- I call the observation from Michela earlier, right, it's from OT attacks can come from sort of the IT side into the OT side. A couple examples. one is the Ukraine attack back in 2015. We all know that. It's about 200,000 people lost power for a period of time. Right? It started from a phishing compromise from the IT side because of a lack of security control, like proper segmentation between IT and OT. Eventually the attacks was able to come from an IT side into the OT side. And then they were able to shut out some of the OT controllers to create a big impact as a result. So these are some of the examples that speaks to how the attacks can come in. The other recent example is about these -- we just heard, you know -- we saw some guidance from our research side as well. There's a Frosty Goop attack. Again that's impact sort of the energy company in Ukraine in the past quarters. Again over here it's about I believe one of the possible ways about some kind of unsanctioned remote access and the attacker was able to get in and then they were able to again sort of penetrate deeper into the network and then they were able to eventually, you know, touch the OT process and create an impact. When this impact happens different from IT attacks you will see a bigger impact in terms of, you know, disruption of operation. It could be even safety implications.

David Moulton: How frequent are cyber attacks on OT environments according to some of our recent data?

Michela Menting: So they're very frequent unfortunately. You know cyber attacks against OT environments, you know, they happen on a monthly basis. You know the research definitely showed that monthly was probably the recurring, the highest kind of point of interaction with threat actors. But we saw quite a bit also on a weekly basis even and definitely quite often at a minimum on a quarterly basis. So this is not something that's once in a while or some operators. This is, you know, the vast majority on a very frequent basis.

David Moulton: So if I'm a plant operator or I'm trying to provide power or water for a city those are my primary concerns, but what you're saying is that maybe on a weekly, definitely on a monthly, basis, perhaps even more often, you're seeing some level of attack, a significant level of attack trying to come in and disrupt operations, stop that from -- you know, stop that team from being able to deliver what they're looking for. What are some of the things that those organizations are changing or implementing to start to protect themselves? You know as a guy who likes to have the power on and the water on here in Texas where it's very hot, you know when I hear that we've had another cyber attack here in town that is impacting things I'm always scratching my head of what does that mean for me and what are they doing about it.

Michela Menting: I mean that's a good question. I think the mindset is changing a little bit. We're still in a phase where a lot of industrial operators cling on to the belief that their systems are air gapped or their security threat security right? We've got to forget about that. That's -- those days are gone. And they didn't even work that well before. Now there's a very lucrative opportunity for especially cyber criminals to target industrial operation centers. And, you know, Qiang mentioned it just previously certainly from a manufacturing perspective. I mean that's the -- a really successful very hot ground for cyber criminals. Right? There's, you know, some very high economical stakes. You know there's low tolerance for downtime. You know a lot of manufacturers certainly don't have that regulatory pressure for security that critical infrastructure has. So there's a lot of factors that tend to make industrial operations kind of open hunting ground for threat actors unfortunately.

David Moulton: Yeah. A juicy target.

Michela Menting: Exactly. [ Music ]

David Moulton: Qiang, what is it about OT environments that is causing them to be increasingly targeted by cyber threat actors?

Qiang Huang: Yeah. So definitely if you look at these OT environments on one side for manufacturers that's where they make their revenue in terms of critical infrastructure. That's where they're running mission critical operations. That's why often it could be a high occurring target for attackers. So we could break it into a few bucket. One is really financially motivated. Right? We just talked about for it could be arranged to the tens of millions when you get hold of these OT assets and then you spread some as a way to get, you know, financial motivation achieved. It could be for some of the high tech manufacturing this could be espionage to steal intellectual capital from that environment. On the other side for critical infrastructures we also see motivations in terms of nation state or cyber terrorism that really, you know, create an impact in terms of critical infrastructure for political motivation.

David Moulton: How do IT borne threats typically make their way into OT systems? This is honestly something that I've always been curious about.

Michela Menting: Sure. Well, I think I mean there's two if you're looking at it from a very high level perspective. There's two primary ways. One is, you know, you could exploit vulnerabilities in industrial control systems. Right? Build a zero day around it, you know, do some smart coding and brute force your way in. But, you know, that's complex, requires a lot of skill. It's not everyone that can do that. The easier way and the very common way is through social engineering unfortunately, and that is just immensely popular because it works all the time. Not all the time, but it works more often than not. Right? You have email compromise. You have phishing. And you know threat actors are able to obtain credentials that they then use for remote access. And quite often, you know, it starts in the IT space and then they escalate their privileges and there's a lot of lateral movement that happens until they can hit, you know, those OT assets. But I think increasingly you'll see some of that happen and target directly OT. Right? So they won't even need to go through the IT space to get to those OT assets. So unfortunately today I mean it's still very much, you know, kind of whacking someone over the head for the password other than trying to, you know, crowbar their way in through an iron door or something like that. So social engineering unfortunately is highly prevalent and still highly successful even against OT.

David Moulton: How do I see specific attacks compromise the integrity of OT processes and then what are some of the potential consequences of those breaches?

Qiang Huang: I think in the recent past we observed that malware started to understand sort of the ICS DCS control functions. As a result, they can start to really change the integrity of your process. For example you can do a right command. It tells the physical process to do different things. You can change the register value. We heard about some of the water utility attacks. Potentially you can even change the current level of the water in the public domain. It's really scary. You could download a different program file to the machines to fundamentally tell the machines to do different things. Imagine this is in the refinery plant where you're mixing chemicals. If you mix different portion it could really explode. So all of these are really challenges or potential impact. When it comes to the impact this is also again different from sort of the IT ransomware malware. When it comes to OT there's additional operational challenge and the impact. For example, you know, down time. For example, loss of revenue, penalties. For example the safety concerns we talked about. Even the physical damage to your, you know, OT assets and infrastructure. So these are some of the additional impacts when it comes to sort of ICS specific attacks and malwares.

David Moulton: What about some of the immediate business impacts from an OT attack? What are you seeing, Michela?

Michela Menting: If you're looking at OT operations, right, you have a physical process that's happening. You're manufacturing something or you know you're generating electricity or transmitting it or, you know, treating water. Right? So if you have an attack that's interfering with those operations, you know, at worse you have an immediate shut down. Okay? So you're stopping all operations. And operators typically tend to if they suspect something's, you know, wrong they'll do that preventatively. So you're not manufacturing anything. You're not generating any electricity. You're not maybe processing water. And that can have a direct, you know, immediate impact. And that costs money. Right? You know, you have to remediate, but you're also not making any money because you're not producing the products that you're then going to sell. Right? So there's that immediate financial aspect right there, but then there's everything else at companies. Maybe you're paying the ransomware. Maybe you've got to inform regulatory authorities. Maybe you've got to do a lot of incident response, you know, hire an external company to do some marketing, some response. You need to talk to your customers. So there's a lot of -- there's kind of a long tail as well, financial repercussions that happen kind of after the fact and you know could accumulate over months, you know over years sometimes even. And then if you're regulatory liable it could be, you know, financial sanctions from that perspective as well. So, you know, it could be quite a large gamut of financial penalties that kind of occur from a cyber attack. So it's certainly not cheap.

David Moulton: Right. And all of those things end up in my mind driving to reputational damage whether that's with regulators that no longer fully think you're doing a great job or your customers who don't think they can count on you anywhere in between. Even the employees that are going to come in at that production or manufacturing facility and going, "Well, we can't work today because everything's shut down." And Qiang can you talk about some of the main challenges that occur when trying to secure these OT environments, especially compared to IT?

Qiang Huang: Yeah. Sure, David. We talk to hundreds of these industrial organizations to understand what are the top challenges. It doesn't matter what industry. The top challenge I've heard is about the lack of visibility. I don't know what my OTS is. I don't know the risk. I don't know who's talking to whom. Without that it's very hard to secure what you don't see. That's number one challenge. Then you have to realize that because of these OT operational constraints up times these are assets managed by different team. Often the existing security tools don't quite work well on these legacy OT assets. It often comes down to segmentation. But because of a lack of visibility we see insufficient segmentation or threat prevention. Now there are also new challenges because of this digital transformation, the new way of doing business. Two things I want to highlight. One is sort of how do I, you know, have visibility and the security control for all these remote operations, all these unsanctioned connections? It's a big challenge nowadays. And also how do I secure my private LT and private 5G network? And most of these enterprise folks they're not mobile expert. So when you bring all of these together the industry is also facing a huge amount of complexity when they have to bring multiple separate tools to drive that OT security in an effective way.

David Moulton: Qiang, how are organizations dealing with this convergence that is occurring between their IT and their OT systems and security?

Qiang Huang: It's a good question, David. I think obviously when we look at this there's people, process, and technology. Right? I think we're seeing more and more industrial organizations that kind of realize that this convergence because of the business needs so they're making changes in terms of how do they drive OT security. But in general I think we're seeing more sort of top down guidance in terms of having consistent security for the OT environment. With that being said, we still see IT OT collaboration. It's still, you know, some way to improve. In our recent survey about 40% of the industrial organizations still mentioned that their IT and OT teams are not fully working together. Only 12% said they're fully aligned. So this is really a challenge that industry needs to solve because your networks are getting more and more convergence and then your networks becoming more flat. So it's a critical element for us to take a look.

David Moulton: That's a pretty big gap if only 12% of those teams surveyed are saying that they're actually aligned on what they're trying to achieve. The threat actor doesn't necessarily care if your OT and IT teams agree. They're looking for that, as Michela put it earlier, you know, I'll say mission impossible way in or more likely the social engineering way in. And that's quite frightening. Michela, you spoke about regulation earlier and I wonder what role does regulation play in driving some of these OT security initiatives.

Michela Menting: Yeah. I mean in critical infrastructure certainly there's been accompanying regulation as that critical infrastructure often has that national security imperative. Right? We see it from a sectoral perspective in energy and healthcare. But increasingly we're seeing industrial operations in general come under the purview of regulation. Certainly in Europe which I'd say is pretty advanced now we've got regulations such as, you know, the Cyber Resiliency Act. We've got an update on the NIS directive. So the NIS 2. And what are these regulations kind of saying? So the Cyber Resiliency Act is basically saying, you know, secure your supply chain. You've got to be resilient against cyber attacks. Or else, you know, financial penalties. You know, personal liability even for some of those executives. NIS 2 expands on what is traditionally understood as a critical infrastructure operator. Manufacturers now come into that purview. Right? So they need to start paying attention to how they secure their operations. You know in the U.S you've got executive orders coming down. We have, you know, the 14028. You know, it includes quite a lot of measures on supply chain security. So, you know, it's kind of this rolling ball. It's going to get bigger. There's going to be a lot more pressure on a lot of industrial operators. You know it starts from critical infrastructure, but soon it's going to encompass a lot of different industrial operators. And, you know, that's the concern and the research has shown that quite a lot of these industrial operators expect regulatory pressure to grow so that's something they need to start paying attention to. [ Music ]

David Moulton: So, Michela, you were the lead author on the state of OT security 2024 report. And I'm hopeful that you can talk to me about some of the key things that you saw in that research, those insights that you're able to provide.

Michela Menting: I mean what really stood out to me was the frequency of cyber attacks against OT networks and OT assets. And the very high number of respondents that experienced a cyber attack. Right? We sort of knew this was happening, but to have it reflected back in the stats it confirmed that in a kind of frightening manner. Right? Over 75% experienced, you know, a cyber attack in the last year. 1 in 4 had to shut down operations. I mean this is not just something that's minor or affects a minority of operations. This is -- this is big and this is significant and this is kind of scary. And I think it's important that industrial operators start paying attention to what's happening because not a lot of them are going to talk publicly about it, but it's happening regardless. And no one's exempt from that. And I think that kind of finding really nailed home what a lot of people in the security industry have been saying for a long time. And you can't ignore it anymore. You really start -- really have to start facing up to that fact that you're probably going to be next. I mean it sounds scary, but that's the truth of the matter today.

David Moulton: So it sounds like it's not a matter of if. It's a matter of when. And because of the importance of these industries and the amount of leverage that the attacker can get there isn't a reasonable person who would say this is likely to slow down. Qiang, what about you? What are some of the things that you really saw that stood out in this report?

Qiang Huang: Yeah. I think it's really great survey to me as a, you know, cybersecurity technology provider. I felt that the survey reinforced some of our observations from some of the customer challenges trend. For example it's interesting to see 70% of respondents think 5G is going to be a big threat vector. About 75% of them think remote access is on the rise for both employees and third parties. So these are some of the trends we also observed here. What's also a big call out for me is that as these industrial organizations are getting themselves ready and preparing themselves for a digital transformation, how do I secure that journey? The complexity I also mentioned earlier was a big call out. I think about 70% of the respondents said that they'd like to drive a bit of -- to consolidation to get ready for this converged network. I felt that we're very well positioned in this area that's aligned with our vision. So I'm pretty excited that on one side we see more challenges. On the other side I'm pretty excited that we can provide strong solutions and capabilities to help our customers to navigate through this journey.

David Moulton: When you talk about this idea that the thing that is exciting is that we're going to be able to drive some innovation to help customers out that is something that across the board in security and I've noticed it working here seems to be the underlying driving mission. And it's good to know that while the threat actors are picking up the pace or maybe targeting more often the OT environments that there's -- there are solutions. There are ways to address this problem. So let me close with my favorite question for you both. Michela, we'll go with you first. If you could tell me, what is the most important thing a listener should remember from this conversation?

Michela Menting: I think the top one is really you need to stop thinking that you're going to be exempt. As an industrial operator, you know, they're complex networks, Qiang said. And visibility is tricky and finding the right solution and deploying the right solution is tricky. But it doesn't mean you can't try and you shouldn't attempt it just because it's difficult. You need to really kind of be proactive about security rather than reactive and waiting for that attack to happen as if you're going to think, "It might not happen to me. I'll wait until the other shoe drops and then I'll start doing something." No. Don't think that way. Think about being proactive. Think about doing something before things get ugly. And I think that's key, you know. You can really minimize a lot of the fallout and the financial pain if you're better prepared.

David Moulton: I love it. It makes a ton of sense. Qiang, what is the most important thing that a listener should remember from this conversation?

Qiang Huang: Yeah. Actually I'd like to highlight two things if that's okay. I think one is really in the light of all this trend make sure that you work with your organization, work with your management, to really have that top down alignment to make OT security a key initiative. That's very important to get to that alignment at your key collaboration. Then from the technology side, right, it's really about make sure you gain the visibility of your OT environment and think about the platform approach to really provide that holistic visibility and security at the same time for multiple, you know, surfaces we're observing. So that's also very important for you to be able to scale for today and also for the future. [ Music ]

David Moulton: Michela, Qiang, thank you for a great conversation today. I appreciate you sharing all of your insights on the OT security and the convergence that you've seen between OT and IT and some of the insights that you brought forward from a great report.

Michela Menting: Thank you, David. Appreciate it. It was -- it was great being here and I think we had a great discussion.

David Moulton: That's it for today. If you like what you heard, please subscribe wherever you listen and leave your review on Apple podcast or Spotify. Your reviews and feedback really do help us understand what you want to hear about. I want to thank our executive producer Michael Heller, our content and production teams which includes Kenne Miller, Joe Bettencourt, and Virginia Tran. I edit "Threat Vector" and Elliott Peltzman mixes our audio. We'll be back next week. Until then stay secure, stay vigilant. Goodbye for now.