Control Loop: The OT Cybersecurity Podcast 9.20.23
Ep 34 | 9.20.23

Don't take energy for granted.


Dave Bittner: It's September 20, 2023, and you're listening to Control Loop. In today's OT Cyber Security briefing, Redfly cyberespionage targets a national grid. DHS Threat Assessment looks at critical infrastructure threats. A look at the ICS threat landscape, DoE grants for research into distributed energy cybersecurity. CISA offers free vulnerability scanning for water infrastructure and CISA issues ICS advisories. Today's guest it Michael Toecker, he's a Cyber Security Advisor at the United States Department of Energy's Office of Cybersecurity, Energy Security, and Emergency Response; we talk Community Defense. The Learning Lab, Mark Urban's conversation with Alex Baretta, Senior Solution Architect at Dragos; they're talking Secure Remote Access. [ Music ] Symantec warns that the "Redfly" threat actor use the" ShadowPad" trojan to compromise a national grid in an Asian country for as long as six months earlier this year. The attack began in February and the objective appears to be espionage or battlespace preparation. The campaign has overlap with previous attacks attributed to the China linked threat actor APT41. "Symantec" Threat Hunter team principle Intelligence Analyst Dick O'Brien told the "Register" that the same command and control server was used in a breach in India's power server last year; the researchers don't make any definitive attributions however. "Symantec" also hasn't disclosed which country was targeted in this case, although O'Brien to "Wire" that it was one that China would have an interest in from a strategic perspective. Symantec notes: "While Symantec has not seen any disruptive activity by Redfly, the fact that such attacks have occurred in other regions means they are not outside the bounds of possibility. The U.S. Department of Homeland Security's annual Threat Assessment warns: "That state backed actors are exploring the use of artificial intelligence to assist in writing malware for disruptive attacks against U.S. critical infrastructure," "CyberScoop" reports. The reports states: Malicious cyber actors have begun testing the capabilities of AI developed malware and AI assisted software development; technologies that have the potential to enable larger scale, faster, efficient, and more evasive cyber-attacks against targets including pipelines, railways, and other U.S. critical infrastructure. Adversarial governments, most notably the PRC are developing other AI technologies that could undermine U.S. cyber defenses including generative AI programs that support malicious activity such as malware attacks. Kaspersky had published a report looking at the threat landscape for industrial automation systems in the first half of 2023, observing an increase in attacks against ICS entities in Western Europe, the U.S. and Canada, Australia and New Zealand and Northern Europe. Despite this increase however, these regions still have the lowest ICS percentages compared to other parts of the world; Africa, Southeast Asia and the Middle East had the highest percentage of ICS attacks based on Kaspersky's visibility. A survey by "Cybellum" has found that 70% of industrial equipment manufacturers have immature product security programs, While nearly half lack a dedicated security function for control systems and devices within their organizational structure. The researchers note that the primary product security challenged faced by industrial equipment manufacturers revolves around the pursuit for enhanced efficiency, including optimizing the utilization of cybersecurity talent, reducing manual efforts, and expediting processes. The U.S. Department of Energy has awarded $39 million dollars in funding for nine National Laboratory Research Development and Demonstration projects focused on cybersecurity for distributed energy resources, "Security Week" reports. The money will go to projects at Argonne, Brookhaven, Lawrence Berkeley, Sandia, Oakridge, Pacific Northwest and the National Renewable Energy Laboratory. The DoE said in its announcement, "The National Laboratory teams aim to improve real-time DER Operation data analytics using Artificial Intelligence, machine learning and secure cloud-based solutions for DER applications. The labs will develop security solutions for current and emerging communication architecture systems and develop innovative real-time or offline analysis technologies that secure DER. Last week, ransomware attacks against two large casino operators, Caesar's Entertainment and MGM Resorts attracted widespread attention. While slot machines are hardly critical infrastructure, perhaps the incidence will draw more attention to the risk of consequential attacks. "WIRED" argues that while there seems to be an element of frivolity in the attention high profile incidents, like the attacks against MGM Resorts and Caesar's Entertainment attract, nonetheless, such attention drives awareness, response, and sometimes affective public policy. "WIRED" quotes Lesley Carhart, Director of Incidence Response at Dragos, which specializes in industrial cybersecurity as stating, "Attacks against casinos are dramatic and draw attention; we have whole movie and TV franchises about casino heists. A lot of life-impacting attacks on critical infrastructure and healthcare occur far less visibly, and therefore, they aren't an easy draw for mass media. I do not think this is an issue with cybersecurity or even media in its entirety, it's a human psychology issue. We've had that problem for a long time in the industrial control system cybersecurity space where attacks could really mean life or death but are not a great story." There is certainly an ongoing ransomware threat to infrastructure. In one current example, the International Joint Commission an organization that handles water issues along the Canada/United States border has experienced a ransomware attack. The commission has disclosed few details. The" NoEscape" Ransomware gang claimed responsibility of the attack saying it's taken 80 gigabytes of sensitive data, which it will begin leaking if the demands aren't met. The data is said to include contracts, legal documents, personal information belonging to people associated with the ICJ, financial data, insurance information, geological files and much other confidential and sensitive information. NoEscape said it its leak notice, sounding like a comic book villain, "If management continues to remain silent and does not take the step to negotiate with us, all data will be published. We have more than 50,000 confidential files and if they become public, a new wave of problems will be colossal. For now, we will not disclose this data or operate with it, but if you continue to lie further, you know what awaits you." Release of business data is troubling in itself the possibly of pivoting from business systems to operational technology proper, is a matter of even more concern. The U.S. Cybersecurity and Infrastructure Security Agency is offering a free vulnerability scanning service for water and waste water utilities. The agency stated: "CISA uses automated tools to conduct vulnerability scanning on your external networks. These tools look for vulnerabilities and weak configurations that adversaries could use to conduct a cyber-attack. CISA's scanning provides an external nonintrusive review of internet accessible systems. The scanning does not reach your private network and cannot make any changes. CISA will send you weekly reports with information on known vulnerabilities found on your internet accessible assets, week-to-week comparisons and recommended mitigations. Interested organizations can sign up for the service by emailing CISA. And finally, CISA last week issued seven advisories for vulnerabilities affecting industrial control systems; six of the advisories affect Siemen's products, while one relates to Rockwell Automation's Pavillion8 Predictive Control Software. [ Music ]

Dave Bittner: Our special guest this week is Michael Toecker, he's a Cybersecurity Advisor at the United States Department of Energy's Office of Cyber Security, Energy Security and Emergency Response. Today we discuss Community Defense. So let's start off with getting to know you a little bit. Can you give us a little bit of your career journey here? I mean, where did you get your start and what led you to where you are today?

Michael Toecker: Yeah, sure thing. I graduated from the University in Missouri at Rolla back in 2005. And I ended up going to work for a top five power engineering firm, Burns & McDonnell. And so, the way that I got -- they had a little startup group within Burns & McDonnell called The CIP group, C-I-P, and you had to spell it right in order to be employed by them. And so, I, was brought into that particular startup group, in fact, I got my start there because I did, a vulnerability assessment on their website after doing my first interview and I sent them the results, so they decided to hire me afterwards. This is not career advice I suggest anyone follow these days, it's a little bit more aggressive than it was back 2005.

Dave Bittner: And so, where did you go next and what ultimately led you to the DoE?

Michael Toecker: Yeah. So, I worked at "Burns & Mac" for about six years doing vulnerability assessments, pen tests and a lot of compliance related activities on electric power infrastructure, and it was pretty much all electric power infrastructure. So I started, cyber in ICS and OT like when it was really not cool and people made fun of it. So, after that I worked for a, utility for about a year, year and a half in their power generation division working on some of the big iron power generation systems, so turbine control systems, balance of plant. I was in hard hat and safety shoes for about 50% of my time working on the cyber security controls for those particular systems. And so, that ended up being a good, set of experience for me there. I went to go work for Dale Peterson and Digital Bond. Soon after that, I ended up going to one of his conferences down in Miami and having a really good conversation with him; we'd only interacted over email before and he was like, "You know, you should come do this for me for a little while. And I went, "That sounds like fun, I can go do that." Ended up being one of his hand-tap minions for about two and a half years until 2014. That's when I decided I was going to try owning my own business for a little while and started up a really small consulting firm, i.e. just me and working with, and working with other contractors like me doing vulnerability assessments, pen tests, design work. Ended up doing a lot of work in the nuclear sector, commercial nuclear on some of their cybersecurity controls that were necessary, just associated with NEI 08-09, that's like working in an entirely world at that point. Left nuclear work for the long part and ended up working for Idaho National Labs, and specifically I ended up working on the DARPA RADICS project, which was the Rapid Attack Detection, suddenly I forget the I, Characterization Systems, component, basically a very large project that was basically developing technology to assist energy sector and the nation at large in, basically assumed breach, right, assuming that the adversary had become so strong that they had bulldozed through all of the particular defenses, how do you build everything back from the ground up? And I was the liaison to the Department of Energy before Caesar at the time. And the idea here was, is to engage private sector folks, right, the folks that actually know how these systems are run because they've been working on them. You know, they probably had hands-on in the system no more than a week earlier at that point when I was talking with them. And they were able to tell you how they were connected and what the consequences were, and most especially how they operated and how they would recover from a cyber incident. And so, it was often a back and forth between the DARPA folks saying, "Oh, have you considered this from a threat perspective?" And them going, "Yes. Have you considered this from an operations perspective? "Like, "You can't physically do that." And have DARPA folks go, "Oh, we didn't realize that. Have you considered this?" And then have them go, "Oh, we hadn't considered the threat might be able to do that." And then, after that it was a natural continuation, to move from this small scale experiment exercise technology development project, into more of an ongoing relationship with the sector and with government itself on cybersecurity for energy systems, and that's how I ended up becoming a federal employee and working within Caesar at that point.

Dave Bittner: What do you suppose it is that attracts you to, this side of things, to the energy side?

Michael Toecker: I have always taken energy for granted before I went to go work at "Burns and Mac; the ability to be able to turn on a light switch and have lights come on, it was just something that happened, it was expected behavior. And when it wasn't suddenly, you know, when it wasn't doing the expected behavior, when it wasn't coming on, it was always jarring at that point. But coming into and working at "Burns and Mac" and seeing how everything is put together and how everything relies on energy and how we are using, you know, these computer systems in new and different ways, and how they're incredibly important to, you know, maintaining, operations and reliability, it really kind of found a niche in my brain that went, "You know, I'm not someone who can live completely inside his head, inside of like a data structure that only exists inside a computer, but I'm also not someone who can just, abandon tech all together, and I found a nice combination of the two in there. And then, the idea that I'm, you know, helping keep the lights on, you know, we have a tendency to use, that phrase when talking about electric power, you know, it sums up a lot of different things; you know, keeping the lights on, keeping, the world moving, keeping water flowing, keeping water treatment going; keeping hospitals in energy so that they can, you know, keep people alive. And, you know, all these other things that support our modern way of life, I really found my niche in there, so.

Dave Bittner: Yeah. I mean, it's literal and metaphorical.

Michael Toecker: Yes, especially when you're working around it and you have the turbine engineer from, you know, x many years ago saying, "Okay, Mike, when it starts rattling like that, you don't want to be at its side, you need to come, in front of or behind the turbine because the blades have a tendency to come out that direction. You know, it also brings a nice dose of reality back to you too.

Dave Bittner: Yeah.

Michael Toecker: They call it "blade liberation."

Dave Bittner: Right. How about these days at the Department of Energy, what is your day-to-day like there? Michael Toecker: So at the DoE I am a program manager, effectively. So I have three primary jobs; number one, I serve as a frontline subject manner expert on industrial control systems used in the energy sector. And I provide that experience to whomever in the department needs it at that time. So I'll talk with anybody from Renewables folks' I'll talk to Grid Development folks, I'll talk to folks within Caesar who are working on policy related items. And I basically provide that experience where it is. And if it exceeds a certain level, then the idea is, is that I assist in getting additional experience from our partners at the National Labs or going out to Private sector to get additional capability at that point. So I'm kind of front line, in order to answer basic questions, and then you know, if it needs to be much, much deeper I help in scoping projects and things like that. Two other things that I do is that I'm also the public sector side, of the Cybersecurity Risk Information Sharing Program, which is the program that the department has that takes in data voluntarily shared from, large utilities and utilities that are part of pilot programs within the department. And it takes in network perimeter data and then it goes back and it does a series of analysis and then provides that analysis back to the folks who have voluntarily shared that data, and it's often on threats or vulnerabilities, or, you know, we're starting to see trends, you know, for scanning or for other things. And so, it's been a really great experience working with that, because the CRISP program is unique in government in that it's not one, whole government. All right this is industry leaning forward and saying, "You know, this is something that we want to do collectively and we feel like our government partners can assist us with it." And this is all through the ISAC at this point, which brings up the second part. They've set up a private sector component of it as well, that runs the program and administers the program, and it has an extremely vibrant governance and user group that works pretty diligently on improving the program; on making sure that the program is getting good returns for its data. You know, voluntarily shared data is not compelled data, all right, you just don't give it to the government. The government needs to come back and say, This is the benefit you are receiving for it," all right, and we have to work on that as well. And then, the last component about it is I'm the technical lead for a new pilot program that the department has been working on and continually scoping and bringing to fruition, which is the Energy Threat Analysis Center pilot. And the, idea here is that, this project is, so the CRISP program is a lot of we receive data and we push things out, all right? The ETAC is about bringing in data but also bringing in experience, what I refer to as "energy sector context," and bringing that into the government's sphere so that folks who are familiar with threats, folks, who have a large understanding within the government of, you know, what we're facing, we marry that up against energy sector context and we can say, "Okay, what else can we pull from this?" We're not just looking from one side of the equation, we're also looking at the implementation side. And industry likes this because we move away from this, you know, one vulnerability will own them all kind of discussion or hey, this is an Achilles heel or things like this, and it gets into a more nuance conversation of real security measures, okay, we have multiple layers of defense; we assume breach, right, we work on these particular types of security controls on a consistent basis and we're continually moving forward, you know, on this, that, or the other; we're keeping track of threats. And then, on the government side, the DoE has got a leg up in this, all right, the department owns, what are called like the federal Power Marketing Agencies, which are effectively federal utilities. So we actually own and operate, infrastructure, all right, stuff that requires a hard hat and safety shoes. And not a lot of other federal departments do that, but we all run email; we all run FTP servers; we all have SharePoints; we all have cloud, you know, et cetera, but we all don't have OT. And, one of the things that the energy sector brings us is this better understanding of OT, so that when we're working on issues of concern, or active threats that, we're working together on it or we're not making assumptions. Okay, we're moving forward together in a much more consistent and effective basis. And, that's the idea of the ETAC; it's a lot more complicated because there are a lot more people and when you add a lot more people to a problem it tends to get really complicated really quick. But that's our guiding light at that point is working together on these problems with information that we mutually have. That's a really interesting insight particularly, that you all have as you kind of say, you know, a big iron; you have stuff that's actually running, and not every department in the government has that.

Michaels Toecker: Yeah. There are different places that have it; Department of Energy's got one of the larger concentrations. In fact, the Strategic Petroleum Reserve is under Caesar now and it has, oil and natural gas control systems that are associated with its operation as well. There's also Bureau of Reclamation and the Army Corps of engineers, you know, they operate power generation facilities as well. You know, there are pockets of this within the federal government, but when it comes down to it there's still a lot of, there's a lot of places where, there's a lot of places where context from industry can really benefit work that the government is trying to do to protect, you know, from, large, nation-state type threats.

Dave Bittner: Our thanks to Michael Toecker from the United States Department of Energy for joining us. There's more to our conversation which you can hear in our next Control Loop episode. [ Music ] 

Dave Bittner: In this week's Learning Lab we hear Mark Urban speaking with Alex Baretta, Senior Solution Architect at Dragos; they talk about Secure Remote Access. [ Music ]

Mark Urban: I'm Mark Urban, it's another Learning Lab today; we're going to talk about Secure Remote Access. And for that I am joined by Alex Baretta, he's a Senior Solution Architect at Dragos. Welcome Alex.

Alex Baretta: Thanks for having me, Mark, really glad to be here.

Mark Urban: First of all, can you tell me a little bit, what is a Solutions Architect? What kind of work do you do?

Alex Baretta: Yeah, absolutely, Mark. So a Solutions Architect at Dragos has several facets of our role. Really part of it is going to be, a presales engineer kind of role where we're consulting, with prospective customers, working with them to determine is the Dragos solution right for them? What makes the most sense? What are really the goals that they are trying to accomplish with a technology like the Dragos platform? And then, once that prospect becomes a customer, working with them to deploy, to manage the platform and help them really get value from the product.

Mark Urban: Excellent. And in the course of that, you know, we started talking about Secure Remote Access. And that seemed like, an area that as you work with customers you've seen pop-up in their consciousness. What is SRA? By the way, Dragos does not offer a SRA technology. So, Alex, tell me, what is Secure Remote Access?

Alex Baretta: Yeah. So Secure Remote Access really is a means that organizations can leverage to access resources, internal to a particular site, you know, remotely. And it really became more and more prevalent during the COVID-19 pandemic as remote work kind of went on the rise. And it became really prevalent in a lot of industrial organizations as folks realized that a lot of the resources that were performing maintenance needed to continue to have, that access, but weren't able to go onsite due to the restrictions. So, what a lot of organizations did is they either implemented some sort of, you know, remote connection, whether that was done via an RVP suite; whether a vendor was able to put in a remote access box directly to a device. But a lot of those solutions, are not secure, right? And so, what a lot of organizations are looking for is something that's going to allow these internal and external 3rd party resources to access their environment and the resources they need in a more secure manner and Secure Remote Access.

Mark Urban: Got you. So I'm working from home, I'm in charge of managing one of the control systems in my plant; I want to fire up my software, those aren't necessarily the most secure ways to tap in, you know, from home to the plant and manage the system's, so secure mode access kind of comes in over the top of that to make that, a secure connection. So it's a popular topic because there are more people doing this, accessing their systems remotely from home, and needing a way to make sure that that's protected. Is that why it's, you know, as we look at the SANS five critical controls for OT cybersecurity, Secure Remote Access, is one of those five critical controls, why is that? How does it make the cut as to one of the five controls that you want to put into an OT security environment?

Alex Baretta: It really definitely warrants its spot up there because as we get to, kind of the more threat intelligence evaluations of how adversaries are leveraging the existing infrastructure to gain initial access and wreak havoc within these environments, what we've seen emerge over the last several years is that, insecure remote connections, are one of the top threat factors that these adversaries are taking in order to gain that initial access point. And in fact, we've seen quite a bit of ransomware, enter an environment and propagate through these insecure remote access connections. So part of the five critical controls for ICS Cybersecurity, SANS decided to add, that Secure Remote Access piece. Those five critical controls kind of come together to, to guide OT teams and OT teams that are responsible for OT cybersecurity, and give them recommendations into how to operate an OT environment in a more secure manner. So that Secure Remote Access becomes a critical piece of facilitating this environment's functionality and the ability to operate on a defined schedule, because when we really look at operational technology, the most important and most critical aspect of that is going to be the availability of the office to perform their intended function. And if they're not able to perform that intended problem then, well, we have a problem. So, having Secure Remote Access be on inside [phonetic] critical controls ensures that maintenance is able to take place on an expected basis, on a scheduled basis; change management is able to be followed as expected, that organizations really are going to be able to monitor and continue operations within this environment in a secure manner.

Mark Urban: Got you. Now, Secure Remote Access had been around for a long time. And, I'll bring you back; I've been in this business for a while. I remember back in, wow, this is going to date me, back in the late 1990s, when I was in charge of the computing network. I was installing a Shiva remote access bank of dialup modems with a secure I.D. token card and that was my first brush with Remote Access Technology. Things have come a long way since, you know, since the Stone Age, but tell us a little bit about, you know, some of the technology names that kind of swirl around, you know, Secure Remote Access. Alex Baretta: Yeah. And so, Secure Remote Access, personally, I've been using it to kind of define a more generic term, but there's, there's quite a few ways that it can be not defined, a few terms that might be turned around, that all may have some sort of a Secure Remote Access aspect. Some of those might be IP Sector PNS, Zero Trust Network Access, Privileged Access Management. All of these are different ways of saying a similar thing; they're using similar tools in order to accomplish a secure way of remotely accessing an environment. And all of those products and tools that I've just listed have multiple other use cases. And, I don't mean to say that, you know, they're all Secure Remote Access technology, but a lot of them have similarities to Secure Access Technology; there's a lot of overlap with those tools which can be used to accomplish Secure Remote Access when we look at it from an OT perspective. So, yes, it has been around for a pretty long time; I'm sure a lot of IT organizations are going to be pretty familiar with Secure Remote Access. And we'll touch on, you know, when we talk about picking a Secure Remote Access Vendor, a lot of folks that might have something in place in IT, it might makes sense to just overlap that into OT, or extend that deployment into OT because it's going to be, the fastest and most effective way to kind of gain that security in an OT environment. So when we think about the history, you know, it really started to change in COVID-19 and along with the, evolution of ICS attacks over the last probably 5 to 10 years, that's when we really started to see the focus on threats targeting the OT environment directly. So that's kind of why we've seen a lot of folks in the OT space look at Secure Remote Access in a different light and take it a little more seriously. Thanks very much, and again, we'll provide the link to the, blog in the show notes. And, that's it again for today's Learning Lab. [ Music ]

Dave Bittner: And that's Control Loop brought to you by the Cyberwife and powered by Dragos. For links to all of today's stories check out our show notes at Sound design for this show is done by Elliott Peltzman with mixing by Tré Hester. Our Senior Producer is Jennifer Eiben; our Dragos producers are Joanne Rasch and Mark Urban. Our Executive Editor is Peter Kilpe. I'm Dave Bittner, thanks for listening. [ Music ]