Control Loop: The OT Cybersecurity Podcast 10.19.22
Ep 11 | 10.19.22

An IT security professional walks into an OT bar.


Dave Bittner: It's October 19, 2022, and you're listening to Control Loop. In today's OT cybersecurity briefing, an assessment of port and terminal cybersecurity in the U.S. Tata Power discloses a cyberattack. The White House issues statements on cybersecurity. India's power company collaborates on energy sector cybersecurity. Our guests today are FBI Baltimore special agent in charge, Tom Sobocinski, and supervisory special agent for cyber, Tom Breeden, sharing the FBI's collaborative approach to working with industry. And our Learning Labs segment is the first in a series with Mike Hoffman, a principal industrial consultant at Dragos, teaching infosec professionals how to think about OT security. This episode discusses the fundamental differences between IT and OT security.

Dave Bittner: U.S. law firm Jones Walker LLP has published the results of a survey on the cybersecurity of ports and terminals in the United States. The study looked at blue water facilities, adjacent to the open sea, and brown water facilities, usually located on inland rivers. The survey found that 90% of respondents believed their organizations were prepared to defend against cyberattacks, but 74% said their systems have been subjected to breaches or attempted breaches over the course of the past year. The report notes that these organizations need to be preparing themselves to comply with the U.S. Cybersecurity and Infrastructure Security Agency's new requirements. 

Dave Bittner: The report states, Marine facilities need to be aware of the Cyber Incident Reporting for Critical Infrastructure Act, CIRCIA, that was enacted in March 2022. CIRCIA requires CISA to develop and implement regulations requiring a company that operates in one or more of CISA's 16 critical infrastructure sectors to report covered cyber incidents and ransomware payments to CISA within 72 hours of the company's reasonable belief that a cyber incident has occurred and to report ransom payments within 24 hours after a payment is made. These new authorities are regulatory in nature and require CISA to complete mandatory rulemaking activities before the reporting requirements go into effect. CIRCIA mandates that CISA develop and publish a notice of proposed rulemaking, which will be open for public comment and a final rule. 

Dave Bittner: CIRCIA also requires that CISA consult with various entities throughout the rulemaking process, including risk management agencies in CISA's critical infrastructure sectors, the Department of Justice and other appropriate federal agencies, and a soon to-be-formed DHS-chaired Cyber Incident Reporting Council. As of the date of our report, this work is underway. Each facility should consult its legal counsel for the latest developments in this process. 

Dave Bittner: The report also calls for more training for employees and collaboration between organizations in the industry, stating, the Jones Walker 2022 Ports and Terminals Cybersecurity Survey found that only 24% of brown water ports and terminals required staff to participate in annual training. As an association dedicated to fostering mutual support among our members, we were also concerned to learn from the Jones Walker survey that 25% of the respondents still do not collaborate with others in the industry to improve cybersecurity efforts. It seems so obvious that one way to thwart cyberattacks is to share best practices and to collaborate with each other across our industry. Industry associations are ideal for this, especially for the smaller facilities along our nation's inland waterways. 

Dave Bittner: Indian energy company Tata Power disclosed on Friday that it was hit by a cyberattack that affected some of its IT systems, The Record reports. The nature of the attack is unclear, but the company says its operational technology is still functioning. Mint quotes Tata Power as saying the company has taken steps to retrieve and restore the systems. All critical operations systems are functioning. However, as a measure of abundant precaution, restricted access and preventative checks have been put in place for employee- and customer-facing portals and touch points. The company will update on the matter going forward. 

Dave Bittner: The Economic Times cites a senior official from the Maharashtra Police's cyber wing as saying that an intelligence input had been received about threat to Tata Power and other electricity companies. The officials said the companies have been alerted to the threat. The indications are that the incident involved a cyberattack against Tata Power's IT systems, but it bears close watching for any potential effects on or pivot into control systems. 

Dave Bittner: The sabotage of the Nord Stream's pipelines remains under investigation. TASS has stated that Russia is displeased that Gazprom has not been and no doubt will not be invited to participate in the inquest. The Ministry of Foreign Affairs called the Ambassadors of Germany, Sweden, and Denmark onto the carpet for a dressing down. TASS quotes the ministry as explaining it was stressed that if Russian experts are denied access to the ongoing investigation, Moscow will assume that the above-mentioned countries have something to hide or that they are covering up the perpetrators of those terrorist acts. Naturally, Russia will not recognize any pseudo-results of such an investigation unless Russian specialists participate. 

Dave Bittner: Physical sabotage inevitably raises the possibility of cyber sabotage. Addressing an energy conference in Moscow, President Putin pointed with grave and statesmanlike concern to the Nord Stream sabotage as an example of a growing trend toward terrorism directed against infrastructure, The Telegraph reports. The sabotage sets, he said, a dangerous precedent, something the global community should fear and take steps to address. The sabotage shows that any critical, important object of transport, energy or utilities infrastructure is under threat. He wasn't concerned at the conference to name culprits - merely to sound a warning. He has blamed NATO or the U.S. for the sabotage. Almost everyone else suspects Russia. 

Dave Bittner: The White House has issued a statement on the steps the Biden administration has taken to improve cybersecurity for critical infrastructure, stating the administration has worked closely with key sectors, including transportation, banking, water and health care, to help stakeholders understand cyberthreats to critical systems and adopt minimum security standards. This includes the introduction of multiple performance-based directives by the Transportation Security Administration to increase cybersecurity resilience for the pipeline and rail sectors, as well as a measure on cyber requirements for the aviation sector. Through the president's National Security Memorandum 8 on improving cybersecurity for critical infrastructure security systems, we are issuing cybersecurity performance goals that will provide a baseline to drive investment toward the most important security outcomes. We will continue to work with critical infrastructure owners and operators, sector by sector, to accelerate rapid cybersecurity and resilience improvements and proactive measures. 

Dave Bittner: The statement adds, this month, we will bring together companies, associations and government partners to discuss the development of a label for Internet of Things devices so that Americans can easily recognize which devices meet the highest cybersecurity standards to protect against hacking and other cyber vulnerabilities. By developing and rolling out a common label for products that meet U.S. government standards and are tested by vetted and approved entities, we will help American consumers easily identify secure tech to bring into their homes. We are starting with some of the most common and often most at-risk technologies, routers and home cameras, to deliver the most impact most quickly. And, needless to say, many of the labeled IoT devices will end up in industrial environments. If you think of the familiar Energy Star labels on products that consume electricity, the anticipated labels can be expected to look something like that. 

Dave Bittner: India's Science Wire reports on a collaboration between India's government-owned Power Grid Corporation of India Limited, the Indian Institute of Science, and the Foundation for Science, Innovation and Development to improve cybersecurity for India's power grids. The entities are working together to form a think tank called the Power Grid Centre of Excellence in Cybersecurity in Power Transmission and Grid Operation. Shri K. Sreekant, chairman and managing director of Power Grid, stated, cybersecurity in transmission and grid operations is critical in today's digital era. Development of robust defense against cybersecurity is of paramount importance in maintaining reliable power supply. Continuous and collaborative research involving academia and industry for development of cyber resilience systems, as well as capacity building, is the need of the hour toward creating a safe and secure grid. Sreekant added that Power Grid, the largest transmission utility in the country, is happy to associate with the Indian Institute of Science, Bangladore (ph), for setting up the Power Grid Centre of Excellence in Cybersecurity. 

Dave Bittner: The U.S. FBI is actively engaged in outreach with businesses of all sizes across the nation, bringing their resources and expertise to bear to help defend against cyber threats. I recently met Thomas J. Sobocinski, special agent in charge of the FBI Baltimore Field Office, and Supervisor Special Agent Tom Breeden, who heads up cyber operations at the Baltimore Field Office. 

Tom Sobocinski: The FBI - obviously, we have been around for over 100 years now and have a really robust background in investigations and collaboration, both with our federal law enforcement partners and state partners, but also with corporations. And so using those skills, we were and are continuing to leverage that now in the cyber realm. And I think that it is obviously growing and will continue to grow. And things like this podcast allow us to have that conversation with a wider audience. 

Dave Bittner: Tom Breeden, in terms of the actual cyber part of the mission - that specialty - where do you plug into that? 

Tom Breeden: From the cyber point of view, you know, I think there is sometimes the hesitation. You think only the FBI as violent crime or counterterrorism. But we really believe strongly that we have a huge role to play with any organization's cybersecurity program, and particularly from - everything from providing a threat picture of actors, but also, if there's been some activity on the network, that aberration - that strange activity on the network - we believe that we can help any organization provide context to that threat activity and, in essence, beef up their cybersecurity program in general. 

Dave Bittner: My understanding is that a big focus for your organization lately is collaboration - really a two-way street between organizations and yourselves. 

Tom Sobocinski: That's absolutely true. I mean, I think there is a stereotype of the FBI that was well earned for generations of - the FBI is going to come in and take over. And so whether it would be bank robberies back in the '30s and '40s, and then, you know, 9/11 happened. And I think that forced us to work with other entities and other individuals that we never would have worked with before. And I think we were - whether it was pleasantly surprised or just surprised, we found out that made us better as an organization. And so we've now got 20-plus years of really understanding that collaboration matters, and it makes us better. It makes the country safer. And so we now are - that is who we are. That's the culture that the FBI now has. And so as cyber kind of started to come up and become a bigger priority for the FBI, we are now just organically having that become part of how we do business. 

Dave Bittner: Well, help me understand, then - how does that relationship work? If I'm a business, is this a matter of reaching out and introducing myself to my local field office? What's the ideal situation as far as you all are concerned? 

Tom Breeden: There are 56 FBI field offices across the U.S., and there are FBI personnel in U.S. embassies across the U.S. And that's really where we think our strength is. It's our ground game, so to speak. Where, in the U.S., I mean, we have cyber specialists at every field office. And that's in, I mean, everywhere from New York to Maryland to Florida - name it, right? - California. We have agents there that are cyber specialists. If you can - if a business can develop that relationship before an incident happens, it's only going to strengthen their security posture because, when that incident happens, they'll know someone to call. And it won't be like, let me introduce myself. Sometimes there's several layers of legal counsels and cybersecurity teams and firms in between, and that information can go smoothly when those relationships are already established. 

Tom Breeden: SAC Sobocinski mentioned about how far we've come. I remember, when I started working cyber, we would do what we call victim notifications. And a lot of your listeners have - some of your listeners have had an FBI agent knock on their door or send an email or, hey, I want to talk to you about a threat in your network. And there were times we responded with very little information. And there were times when we would - unfortunately, back, you know, a decade or so ago, we would say, we have something in your network. We can't really tell you what it is, but can you look and see if you see anything strange? Those were tough times. Those were hard interactions. 

Tom Breeden: And - but we really - I think we've learned a lot since then. And one of the feedbacks that we would receive - I remember from - some CISOs would say, I love it you came to my door. You're trying to help. I need context of this threat information. And that's what - when you're working with the FBI, when you're collaborating with us, that's - we're going to work as hard as we can to - so your company can be as strong as it could be. 

Tom Sobocinski: Yeah. I just want to add to that. I mean, I think to - going back to the question, which is when do you want to be reaching out to us, it is absolutely before the event. And so we want to have a relationship with you. We want to be providing some of your listeners the information that they need to protect themselves, not to just deal with something negative once it happens. And so it's really important to have that relationship. Now, obviously, we can't do that for everyone at the same level, so there are certain industries that are really important to us. Obviously, clear defense contractors for obvious regions, but then also other critical infrastructure entities are really important. 

Tom Sobocinski: And then there's a third piece that is also important, which is industries that are developing that may be vulnerable to other foreign actors. And that's a piece that is - you know, changes minute by minute. And so, you know, clear defense contractor - obviously, that's classified information. They're storing it in a certain way. They know to protect this. But there are also industries that are creating new and really exciting products, software, things in certain industries that could ultimately be used in a classified environment. They just don't know it yet. And so it's important for us to have the relationships with them so that they know in advance how they can protect this information. I mean, it's pretty clear that this is a growing problem, No. 1, and it's an expensive problem. It's an expensive problem if you are a victim, but it's also an expensive problem to keep yourself from becoming a victim. And if there are ways that we, the FBI, can help you do that, that is now part of our mission. It's what I have Tom and his team doing on a daily basis, not just the reaction to that problem. 

Dave Bittner: Can we - talk some more about that local element because, you know, the FBI has the IC3 for reporting things, and that is a useful way to get that information. But it strikes me that - again, it's that local relationship. If I can pick up the phone and call one of you even just to say something doesn't feel right, that's a much more effective way to get a response than, you know, sending an email off into the ether, right? 

Tom Breeden: Yeah, so that's the proactive sharing. That's the next level, I guess, of, I think, where a security program should be. So it's not just waiting for OK, we got to activate our incident response plan. It's - there's an aberration on our network. We're interrogating that endpoint. In the meantime, let's see if we can get feedback from the FBI on this. And if you've got that proactive relationship - you're right, Dave - that's not an IC3 complaint necessarily. That's reaching out to your local office, engaging that. And then there's a dialogue back and forth. 

Tom Breeden: And again, we're not there to look - continue with Sobocinski's analogy - we're not there to look at the bank robbery. We're not there to look to the lending department - right? - and look for something there. We're there to look for the evidence of the threat actor. 

Tom Sobocinski: And if you're a CISO, you should have the FBI already input into your crisis response plan, whatever that is. If you're the executive over that CISO - if you're the CSO - you need to make sure that we are already in that plan. My next question would be, when's the last time you talked to the FBI, and who is that person that you've talked to? And so I spend a lot of my time meeting with various executives - like, after this, we're going to go meet with another one - on issues just like this. And so we are - we're there for you. We're expecting that dialogue. But also, we need to know if you're interested in it and if it's something that you want to engage with us. So it is absolutely a two-way street. 

Dave Bittner: What do you suppose the future holds as we look, you know, 10 years down the road at the mission of the FBI when it comes to cyber? Where do you all, you know, see yourselves continuing and growing? 

Tom Sobocinski: I think the FBI is the premier investigative agency in the world. And I - I mean, I can say that immodestly, but I really do think we are. And as this space kind of creates more solid lanes in the road, and individuals and organizations start to specialize a little bit more, I think we will remain the investigative expert for this area. And so that doesn't mean that we won't grow, but I definitely do not think it's going to reduce in any way. 

Tom Breeden: And I - just to tack on to that - I 100% agree. I think how that's going to play out - it's going to play out through deeper collaboration, even more so. I mean, we've highlighted a few there. But I think you're going to see more integration with our foreign partners. I think those walls are going to turn into glass walls, and then it may be just barely barriers because the attackers, if they're attacking the U.S., they're probably attacking a Five Eye partner. 

Tom Breeden: I think secondly, you'll see the tech sector, the security tech sector and the FBI - more collaboration there also because, again, there's so much information in both those entities that I think that collaboration will continue. And, you know, I can remember, when I first started working cyber, literally cybercriminals were using Western Union to move money. And look where we're at now. So there's a lot of challenges out there, and I'm confident that we're going to keep growing and keep advancing to be ready to keep pursuing threat actors. 

Tom Sobocinski: Yeah, and we're not unique. I mean, I think the U.S. government in general is growing. And so you have organizations like CISA that didn't exist 10-plus years ago. And so what are they going to develop into? How do we mature with them - not to compete with them, to work with them? I think that space is moving forward in a way I like. I mean, you know, obviously, as I talked about earlier, you know, the FBI - we're going to come take over. This isn't a space that we're able to just do that. And so working with all of these other partners is going to become more and more important as we move forward. 

Dave Bittner: For that CISO who wants to start that relationship, what's your advice? What's the best way to get started? 

Tom Breeden: Yeah, call your local FBI office. If you're in Maryland or Delaware, it's call the local office here. We're here. And we will get you connected to a cybersecurity investigator. And the same throughout the whole U.S. - call your local office. And I think you'll be - it will add to your program, and I think you will - it'll help with your business. 

Dave Bittner: That's Supervisor Special Agent Tom Breeden from the FBI's field office in Baltimore, joined by Tom Sobocinski, special agent in charge of the Baltimore Field Office. 

Dave Bittner: Next up in our Learning Lab, Mike Hoffman, principal industrial consultant at Dragos, teaches infosec professionals how to think about OT security. 

Mike Hoffman: So a little bit about myself - so I have around 20-some years of experience, predominantly in oil and gas. My background really started back when I was doing instrumentation control systems as a technician, coming up through and doing analytical systems leading into automation, SCADA systems, DCS systems, that kind of stuff, working across downstream, midstream and upstream organizations within oil and gas. Now I've been with Dragos for a year and a half as the principal industrial consultant, so a lot of what I do is do a lot of architecture reviews and tabletops and also work with customers in their efforts and the journey along the way as they try to implement better security controls. 

Mike Hoffman: And so let's get in, though, to really the meat of the subject. And that's really the difference between OT and IT. And I would like to take this back often to - you know, we're still back and talking about people process and technology. And a lot of people will say that, hey, convergence is going on. I'd like to say that it's mostly occurred already. So if you're having those discussions around convergence, convergence is gone. The minute we put a Windows operating system within our industrial control systems, convergence was taking place. That's been 20 years ago now. So we've been through this different journey of convergence, if you will. Some industries are way more converged than others, and others are very much isolated still within - when you think about interconnectivity. 

Mike Hoffman: But a lot of times, folks, when they have this background, they begin to thinking about from an IT perspective, can we apply those same IT controls in OT? And so that's a common question I get. Well, since we are running this similar type of equipment, can't I use those same controls? Should I take my, say, same playbook for IT and apply it to OT? And the answer might not be so. There - it needs to be tailored, and we'll be talking about that a little bit more. And then it's also always, where to begin? Where do we begin with our journey? What are the key things that I, myself, and my company need to be focusing on? And so that is what we'll be kind of talking about here. 

Mike Hoffman: I've been on a lot of tours with folks from the IT side. When we go into control rooms or we have a lot of this equipment in place, the question is often - first comes to mind and you hear it, you hear this discussion going on - is that, hey, this control room, this panel, this operator interface looks a lot like IT. It's running Windows. I recognize those applications. So there's always this question about - that's kind of IT asset. 

Mike Hoffman: A lot of times the next thing that pops up is, well, wow. It's - oof. It's running an older OS. I see Windows XP around here, Windows 7, and that's not good. We need to do something about it. So then the conversation goes that - we kind of need, you know, are we patching it? Are we running vulnerability scans on this machine? We need to, you know, put this on an upgrade list. We have to upgrade it. 

Mike Hoffman: And then, finally - and I think the most important question is, how critical is this, by the way? So if we - if this machine gets compromised or if we rip and replace, can we do it, and how critical is it to the process environment? And so, again, a lot of times the focus is on that IT-ish (ph), I should say, asset only. 

Mike Hoffman: But there are some problems when we think about computers as computers in themselves and we don't think about the complexity and the interconnectivity we have with the rest of the OT environment. Some of the mistakes that I've seen is way too much emphasis on vulnerability management, where just so much focus goes into, you know, trying to make sure systems are patched, trying to make sure we're, you know, we have all the vulnerabilities fixed out there. And it's because on the IT side of the house, this is something that people can get their arms around. We can put our scanners in place. We can put automatic patching in place, automatic rebooting and all those kind of things, push out new updates in the IT side. And we have the tools in place, the capabilities, the knowledge. We're good at this. 

Mike Hoffman: On the OT side of the house, we can't do this. In many times and in many ways, our systems can't be rebooted. Our systems may have to stay at a certain patch level for a while. There could be some conflicts. I think of DCOM is just - a patch that Microsoft just came out with, DCOM, that breaks DCOM settings. They can break OPC communication. Those are some of the things that we have to think about a little bit more on the OT side of this patching situation. 

Mike Hoffman: There's also the issue of expecting, you know, OT environments to keep place with IT environments for asset refresh. So when you do see that Windows 7 HMI or HMIs, if you will, in your control room, you may be thinking about when can we lift that over to Windows 10/Windows 11? The answer may be, it is going to take a while because our OSes in the OT side are very much tied to the underlying control system, may it be a distributed control system or a SCADA system, supervisory control and data acquisition, something like that. 

Mike Hoffman: We also - there is a lot of focus on the business side, on the enterprise side, on moving to the cloud. And I'll say on that side, it makes complete sense. On the OT side, most of the environments I go into today have some sort of a cloud connectivity. So cloud is making its in-way (ph) into the OT environments, and it's not all bad. There are definitely instances where cloud absolutely makes sense, but we have to be very, very careful about how we deploy cloud, that - where those connections go from cloud? How far down do they go in the layers of OT? Are they going to an edge where we're - you know, we have controls in place, going from a OT protocol to a more IT protocol at that point? We have encryption going on there. Or trying to do cloud connectivity all the way down to that endpoint device, that remote terminal unit, that PLC - that's where the problem comes in with cloud connectivity. And that's where we have to be very, very careful about our cloud architectures. So there's push to the cloud. It can be a problem. 

Mike Hoffman: There's also an issue around ownership and accountability. If somebody from an IT has been tasked with securing these environments, but who owns these systems? Oftentimes, it's the operations manager at that site who actually owns that system. And then you get into - it's the engineering team that runs that system. So there's oftentimes an area where there's - this needs to be kind of discussed and understood who owns it, who owns the budget for this system. Because oftentimes, security will come in and want to put a tool in place, but security is not going to pay for it. So the business pays for it. So you get this issue of ownership and accountability and those kind of things that needs to be worked out. So even though security may have a good idea to put in place, it's the ownership and the ultimate accountabilities that actually that relies that asset manager. 

Mike Hoffman: So what is most important for OT? It's more important than just confidentiality, integrity and availability and flipping that around on the OT side. Because within the OT side, it's not just this simple. There are other considerations that we need to be thinking about, and the most important is safety. So we have process safety to be thinking about - be at a generation facility or a refinery or water - wastewater. You name that industry. There are safety considerations around keeping that process in place. Also, the equipment has been engineered and designed to run in a certain way. So there's - so there are safeguards in place already, but there's also different things we need to think about from an environmental perspective. We may be dealing with some fairly toxic substance at their sites, and so that may be, you know, a main priority. And then there's also personal safety - keeping our people safe, making sure that they go home in the same way they came into work. 

Mike Hoffman: And part of this, too, you know, where - what is most important? When you think of the triangle - availability or integrity - I would say predominantly, availability is what is - we're focused on. And our systems have been engineered for availability, to keep them operational 24/7. However, in some environments, and even in a processed environment, the confidentiality of some of our systems is actually more important than availability or integrity. Think about pharmaceuticals. Think about, you know, an area where - you know, your favorite soda beverage. The recipe behind that is probably way more invaluable than the actual plant that's operating it because - create another plant. But we can't recreate that PI, that intelligence going on behind the scenes, all that R&D work. So it really depends upon what industry you're in, which is - it determines which one of these factors is more important than the other. But at the end of the day, it's really around that safety aspect. 

Mike Hoffman: At the core of it, are - these systems have to, you know, ensure there are - these OT systems have to ensure that we are continuously delivering these products and delivering these things to our end users. And so when we put security controls on these - in these areas, we need to make sure that at the end of the day, we are supporting these deliverables to our company. We're not just doing security for security's sake. We don't just have a hammer looking for a nail to fix a security problem. 

Dave Bittner: That's Mike Hoffman, principal industrial consultant at Dragos. 

Dave Bittner: And that's "Control Loop," brought to you by the CyberWire and powered by Dragos. For links to all of today's stories, check out our show notes at Sound design for the show is created by Elliott Peltzman with mixing by Tre Hester. Our senior producer is Jennifer Eiben. Our Dragos producers are Joanne Rasch and Mark Urban. Our executive editor is Peter Kilpe. And I'm Dave Bittner. Thanks for listening.