New CISO responsibilities: IoT.

Rick Howard: Back in season one, we started together on this path to explore cybersecurity first principles. The stated purpose for me on why we did this was that I was tired of year after year incrementally improving our cybersecurity posture, but never stopping the pause and think about whether or not we were all going in the right general direction in the first place.

Rick Howard: Oh, if you look back over the last couple of decades, you can definitely draw a straight line of improvement from about the middle of the 1990s to today. The tech has gotten much better, the people are way smarter, and we are all better at our craft. But it doesn't take more than a quick glance at the CyberWire's Daily Briefing (and you can check that out at thecyberwire.com/newsletters/daily-briefing), to determine that the bad guys continue to chalk up impressive wins on the cybersecurity scoreboard. So you have to ask yourself, if we are so much more improved, how is that possible? The entire point of this podcast is to go back to the drawing board and determine just exactly what our first principles are because what we're doing now, ain't working. Clearly. And this episode is a perfect use case. The question is: Who should own the responsibility of securing our ubiquitous IOT environments?

Rick Howard: For years, most CISOs only gave those environments a hand wave in a "we will get to that later" kind of way. But with the Russian GRU, using Ukraine as their personal Petri dish, to learn how to attack industrial control systems and bring down the power grid. See Andy Greenburg's book, "Sandworm", and the Chinese inserting back doors into Chinese made electric transformers. See Joe Weiss's excellent unfettered blog. And, the most recent and almost successful cyber attack against the Florida water treatment plant. You have to wonder maybe protecting our organization's industrial control systems shouldn't be a CISOs additional duty. If they have time. Maybe the responsibility of protecting those environments should be part of the gig, complete with a budget and other resources from the senior leadership team and the board, and brought under the strategic umbrella of the traditional IT security programs of zero trust, intrusion kill chain prevention, resilience, and risk assessment. Hmmm. That sounds like a first principle kind of an idea. Let's find out.

Rick Howard: My name is Rick Howard. You are listening to CSO Perspectives, my podcast about the ideas, strategies, and technologies that senior security executives wrestle with on a daily basis.

Rick Howard: So, what are we talking about here? What is an OT environment? In our last episode, we cribbed from an essay written by Graham Williamson at the Kuppinger Cole Analysts blog. He said that operational technology or OT is a giant set of IT and security activity that manages industrial operations like your electronic door locks in the office. A subset of OT is something called industrial control systems or ICS. These are the systems that if they failed, could cause material damage to your organization and they are very different for every vertical.

Rick Howard: Now facilities managers monitor and manage these OT and ICS networks with SCADA devices or Supervisory Control And Data Acquisition by either continuously monitoring them or by sending data back and forth in batches. They use these things called Programmable Logic Controllers or PLCs to do it. 

Rick Howard: I invited Bob Turner, the University of Wisconsin at Madison's CISO, to the Hash Table and asked him to describe how he thinks about OT in his environment. 

Bob Turner: IoT is a combination of what I call three major disciplines, and the first is alarms and switches and the other control things. That is simply just camera on, camera off, door open, door closed, those types of things. There's not a lot of controls over those because they're binary and PLC-type technology is  usually what we're talking about here, a programmable logic controller technology. 

Bob Turner: The second one are those system controls that are actually taking data back and forth and making adjustments, or reporting tank levels or those other types of things that kind of go beyond the binary.

Bob Turner: The last one are the systems that are critical for life safety and building management type things. One could say that a door open or closed is critical to life safety, but I'm actually talking about our absolute zero pathogen freezers which keep the bugs, where they're supposed to be. I'm also talking about hospital equipment, medical equipment and critical equipment that is informational for police department response, fire response, et cetera.

Rick Howard: Tom Quinn is the T. Rowe Price CISO. When I got him to the Hash Table, he had a slightly different take. 

Tom Quinn: IoT to me is devices are connected to the internet or a network, that aren't related to traditional operating systems for corporations. I use that broad definition purposely because there are so many things that are internet-enabled or IP-enabled  that historically just have not been. 

Tom Quinn: I also separate that out a little bit from something called OT. That's operations or operational technology, and those are, I think, a subset of IOT, but focused on things like facilities-enabled  devices that would also not only include people facilities, but manufacturing facilities, and other kinds of machinery, trains, and the like. There's probably a level of complexity underneath that. 

Tom Quinn: Some are using some flavor of Linux. Some are using some  down Windows version. Some people are using proprietary operating systems. So when you look at it and then that way too, it's a pretty complicated  set of environments and what I've also found is that testing varies as well. The manufacturers of these primarily, machine components and alike these OTs they're much more in the business of selling the machinery, whether it's air conditioning chiller, a power cord, a power switch, that's really what they're doing. And then these internet connections allow them to be managed efficiently and to interact with modern management systems, telemetry systems, and, and alike. 

Rick Howard: For years and I'm talking since the 1990s, OT and ICS security pundits have been criticizing their supporting vendors, the companies that build the machines that plug into the SCADA devices and the programmable logic controllers, for not building security directly into their machines, you know, their hospital, patient monitoring devices and their power grid maintenance systems. But the fact of the matter is that these vendors don't make money by doing that. Facilities managers don't look at the latest crop of Siemens PLCs on sale and say, "Oh, look, I'm willing to pay an extra a hundred dollars per device because it has some security bolted on," if that were true, Siemens would make them. But so far in the last 20 years, that situation hasn't changed. I think we should abandon that path for the moment. It isn't working anyway and consider another strategy that might significantly reduce the chances of material impact. 

Rick Howard: I mean, there are two primary things we are worried about with our OT and ICS environments. One is that a bad guy might be able to degrade or destroy our ability to run our critical systems, similar to what the U.S. and Israel tried to do with the Stuxnet attacks against the Iranian nuclear program back in 2010. The other is that a bad guy might be able to compromise these OT and ICS environments in order to establish a beachhead designed to enable lateral movement out of the OT networks and over to the traditional IT network seeking data they want to destroy or steal.

Rick Howard: I asked Tom if he thought that a traditional zero trust strategy might work here. And if it did, was it sufficient or do we need to do more? 

Tom Quinn: You raised two questions there, right is there a mitigation for segmentation? And the answer is yes. I believe that segmentation of these devices and these platforms is crucial in the corporate environment.  Is there still more to do, right, is it sufficient? And I think the answer is no. 

Tom Quinn: Both of those questions need to be asked and answered and, it's a good dialogue to have. The thing about segmentation though, is it doesn't mean that they can't be compromised. You have to  do vulnerability scanning and the like of these systems. You have to monitor for patches and have them deploy. Then they need to be configured in the right kind of way. And you need things like multi-factor authentication then, like, so there's an operational component of the OTs that I also think CISOs provide a lot of good advice, guidance, and direction for the facilities teams who are operating them.

Rick Howard: Bob Turner agreed. Even if you don't call it zero trust implementing some basic segmentation rules might be a good start. 

Bob Turner: The intention is to create a IoT- segmented network, meaning keeping it logically and as much as possible, physically separated from the regular network, but also making sure that, it's in its own zone for firewalling and it's in its own, area as far as being able to get to the rest of the network components, servers, et cetera. 

Rick Howard: According to the MITRE ATT&CK  Framework, there are nine distinct adversary groups that target ICS networks. Some of the more well-known groups are APT33, Energetic Bear, Bezerk Bear, the Lazarus Group, Oil Rig, and SandWorm.  

Rick Howard: And the security community knows some 95% of the attack sequences for the campaigns run by these groups. Installing prevention controls in our ICS environments seems like a no brainer, but Tom would like to go further. He wants the OT vendor community to do more. He points to a Ripple20 report that came out last year.

Rick Howard: According to Aaron Pohl, a senior penetration tester at CBI, the JSOF Research Lab released a report last year identifying software vulnerabilities with a high chance of remote code execution in a proprietary TCP/IP coding library that has been embedded in millions of OT devices dating back to 1995.

Rick Howard: Yikes. Here's Tom. 

Tom Quinn: And it's also unclear to me, the technology testing and then the cyber testing of those systems two different things. Ripple20 is something that certainly was a shot across the bow for many people. It's not only the good guys that are looking into this. There are a lot of bad guys that are looking into exploiting this area. Ripple20 is a set of 19 vulnerabilities across a variety of OT devices manufactured by a handful of vendors and they allow for some amount of compromise. And whether that is something as simple as a buffer overflow and making the IP address side of the device on stable to full compromise of the IP address side of the device. That could be potentially used as a jump off for traditional corporate devices like Windows or Linux or others. Right? Cause now you have a toehold or they could be used for nefarious purposes for changing the temperature or shutting off power and a whole host of things. So it's a whole host of vulnerabilities underneath that Ripple20 rubric.

Rick Howard: At the top of the show, I mentioned that CISOs have not typically been in charge of securing these OT and ICS environments. They may have been consulted from time to time, but most have not been given the direct responsibility. I asked Bob if he thought that was changing.

Bob Turner: Yeah, it is changing now because of the ubiquitous nature of the network and what goes on it. What I have seen here and when I see from other higher ed CISOs is that there's a transition  in progress to where everybody's now aware of, of those other networks and what they do and they're wanting to bring them under closer control. 

Bob Turner: The modern CISO needs to be thinking about asking those kinds of questions. What do we have on campus here? What is connected? How do we connect the door monitoring systems? How do we connect to the camera systems? Which ones need to have more security than others? 

Bob Turner: I don't know if those questions are being asked in all environments, but I think it's the CISO's responsibility to take charge of security. And you may not be the one who's designing and implementing the IoT environment, but that CISO needs to make sure that environment is secured enough so that a system that is critical to building operations or critical to life safety and security is not compromised because it wasn't designed well.

Bob Turner: Frankly, if the business is heavily dependent upon that kind of technology, the CISO should know about it, and the CISO should have an active interest in making sure that it stays as secure as possible. Those are the things that the CISO needs to be considering when they're wanting to be able to put their face before the C-suite and the board and say," it's not a question of, are we secure, but are we ready? Do we have a structure in place? Do we have a team in place? Do we even have a philosophy in place for securing IoT?"

Rick Howard: Tom agrees with Bob that the responsibility is changing at the very least securing OT and ICS environments is getting more oversight by the CISO, or at least it should be

Tom Quinn: What the CISO has to bring to bear is providing not only a technology observation and consulting capability, but also a cyber consulting capability. And what I've certainly found is the facility teams are open to the dialogue and are willing to partner because they recognize the importance of ensuring the service level of the facilities equipment itself and the ability to monitor it over time. In some shops, I'm sure that there's  various levels of maturity of the conversation between the facilities teams and technology in general and certainly facilities teams and the CISO as well, but it's critical to do that.

Tom Quinn: This is also where the CISO needs to weigh in. CISOs generally do a very good job of industry outreach and communication with the manufacturers of software, PCs and other kinds of things, servers and the like, and we can bring that kind of approach to bear with the facility space too.

Tom Quinn: The goal should be to teach the facilities teams to either operate their technology with the same level of rigor and approach that traditional tech teams do and/or hire people that are going to do that for them. You could make the case that maybe they fall underneath that umbrella of the CISO, or you could take the case that every sets of platforms need people to focus on security and controls. It may not necessarily need to be the CISO  depending on the maturity of the organization and the whole host of things. It may be suitably fine that there are others doing it so long as  people are aware of how they're doing it and they're doing it effectively. The CSO has a role to play in there. How that role unfolds may depend on some of the specifics. 

Rick Howard: As we were sitting around the Hash Table solving the world's problems as we do, all of a sudden, this one idea popped out from all three of us almost simultaneously.

Rick Howard: Why are we still deploying hardware, PLCs and SCADA devices into our OT and ICS environments? Why isn't that just a piece of software sitting in the cloud somewhere that the vendor can easily update. I mean, why hasn't this industry figured out how to deliver their services from the cloud? Why isn't there an ICS SASE service that we all could subscribe to that is connected to a distributed SD-WAN meta layer and runs our ICS network traffic through some kind of security stack in the cloud in a shared responsibility model that will enable us to install our zero trust policy and intrusion kill chain prevention rules, and take the toil of managing that complex environment out of our hands? That is a service we all agreed that we wanted. That is a startup that we all agreed we should begin ourselves.

Rick Howard: Here's Tom. 

Tom Quinn: There's a company called Phosphorus. Chris Rouland is the CEO and founder. He has a scanning technology that is tuned to IoT devices  and OT devices. And there may be others in that space, but I do think there needs to be scanning technology and other management technology put in place to give people a better view of the risks and threats that are in  the IP side of the environment. In addition to all the telemetry that the facilities teams are doing. Who monitors that and looks at the red lights or whatever. It could be your corporate technology team. It may not be, but somebody needs to be  doing that and they need the right tools to do it.

Tom Quinn: I do think it's doable and I also think that segmentation may lend itself  to making that acceptable. Why wouldn't  a facilities service provider I would think that that's a business that they would want to be in. I think the other thing too, is I don't know that the tech teams at most companies want to be in this business either because it's not a laptop, it's not  a Linux server, that's where their sweet spot is. There's a hesitancy for some of these platform teams and corporate tech to want to bridge the gap into facilities tech. 

Tom Quinn: I'm a hundred percent with you that's a real opportunity there for a company. To me, at some period of time, you're going to get Zscaler or  some Proofpoint stack you, you can see this next iteration of consolidation coming. To your point about IoT devices and maybe creating a service provider, I think that is the outcome. I think you'll pay for a security stack for whatever you may have and depending on what your price point is, you'll get certain things turned on or off like a E3 and E5 licensing from Microsoft,  and that you'll have a handful of vendors that do it, and you'll be obligated to do it by regulators.

Rick Howard: Bob agrees with me that all three of us should create this startup.

Bob Turner: Count me in, I think that would be an awesome adventure. It's not the large organizations that are really suffering through a lot of this. It's the smaller ones that just don't have the ability to have multiple teams, but they need to have the multiple types of technologies involved.

Rick Howard: If it isn't obvious after listening to this podcast and probably every episode in this podcast series, I'm a centralist when it comes to securing our digital environments. I believe there should be one person in the organization in charge of all things security: IT, physical, and yes, OT and ICS environments. Not because of I'm an empire builder, which by the way, I've been accused of in previous versions of my security career. But because as a one-stop shop for all things security reduces friction by eliminating bureaucracy and supporting consistency across the board where your material information is stored and your material systems must operate. Frankly, I'm surprised that we've made it to 2021, and this is not a best practice in our community. If you're thinking about first principles, this is the obvious choice to make. And I look forward to your cards and letters where you tell me how wrong I am. 

Rick Howard: That's a wrap as always, if you agree or disagree with anything I've said or anything our guests have said, hit me up on LinkedIn or Twitter and we can continue the conversation there. For next week's show, we are going to have a similar conversation about identity. Why doesn't the CISO own that responsibility? You don't want to miss that.

Rick Howard: The CyberWire 's CSO Perspectives is edited by John Petrik and executive produced by Peter Kilpe. Our theme song is by Blue Dot Sessions, remixed by the insanely talented Elliott Peltzman, who also does the show's mixing, sound design, and original score. And I am Rick Howard. Thanks for listening.