Mentorship, internships, and apprenticeships in OT security.

Dave Bittner: It's August 9th, 2023, and you're listening to Control Loop. In today's OT cybersecurity briefing, the Five Eyes outline the top exploited vulnerabilities. The Brunswick Corporation loses millions to cyberattack. Ransomware in the industrial space. The US Transportation Security Administration updates security rules for oil and natural gas pipeline operators. Our guest, Mea Clift, of Woodard & Curran, shares her perspective on mentorship, internships, and apprenticeships, with an eye on OT security. The "Learning Lab" has the first part of a discussion about the convergence of OT and IT with Dragos' Mark Urban and Kimberly Graham, Dragos' VP of product management. [ Music ]

Dave Bittner: Intelligence services in the Five Eyes last week issued a joint cybersecurity advisory outlining the top 12 most commonly exploited vulnerabilities in 2022. First on the list is CVE-2018-13379, a path traversal vulnerability affecting Fortinet SSL VPNs. Next are the three Microsoft Exchange vulnerabilities, collectively known as ProxyShell. Another note worthy entry was the Zoho ManageEngine ADSelfService Plus vulnerability, enabling unauthenticated remote code execution. The advisory highlighted its connection to an outdated third-party dependency, emphasizing the importance of up-to-date software practices. The widely used Atlassian Confluence Server and Data Center also made the list due to its susceptibility to unauthenticated arbitrary code execution. Governments and private companies relying on this web-based collaboration tool became potential targets. One of the most infamous entries was the Log4Shell vulnerability, which impacted Apache's Log4j library used in numerous products worldwide. The ability to execute arbitrary code and gain full system control made this vulnerability particularly enticing to malicious actors. The US cybersecurity and infrastructure security industry notes that this advisory includes common weakness enumerators or CWEs, which reflect the underlying root causes that led to the exploited vulnerability. CISA states, "In order to reduce the prevalence of common classes of vulnerabilities, this advisory urges technology vendors to implement specific secure by design principles. And to ensure that all published CVEs including the proper CWE identifying the root cause of the vulnerability.” It's worth noting how many of the vulnerabilities continued to be exploited after patches were available. It suggests the effect that slow-patching can have an organization. As CISA so often says, apply updates per vendor instructions, and we might add, sooner rather than later. The Record by Recorded Future reports that a boating manufacturing giant, Brunswick Corporation, lost 85 million dollars due to a cybersecurity incident.The company didn't specify the nature of the incident, but said production was impacted for nine days. The company's CEO, Dave Foulkes said in an earnings call, "The disruption associated with the IT security incident was the most significant in our Propulsion and Engine Parts & Accessories segments. And because of the proximity to the end of the quarter, there was limited opportunity to recover fully within the same period.” He added that, "lost production days on high horsepower outboard engines will be challenging to recover because the production schedule was already full for the balance of the year." Dragos has published its industrial ransomware attack analysis for the second quarter of 2023, finding that 70% of ransomware attacks in the space were against manufacturing companies. The manufacturing sector saw 177 ransomware incidents in the second quarter of 2023. Dragos predicts with moderate confidence that the third quarter of 2023 will witness increased business impacting ransomware attacks against industrial organizations for two reasons. Firstly, the prevailing political tension between NATO countries and Russia motivates Russian aligned ransomware groups to continue targeting and disrupting critical infrastructure in NATO countries. Secondly, as the number of victims willing to pay ransoms diminishes, ransomware as service groups have shifted their focus towards larger organizations, resorting to widespread ransomware distribution attacks to sustain their revenues. Several companies commented on Dragos' findings. Carol Volk, EVP, BullWall thinks that it shows the necessity for industrial organizations to prioritize cybersecurity. A mix of advanced tools, effective employee security training, and collaboration with other companies and the government should be in play. Emily Phelps, Director at Cyware, agreed that ransomware in particular can devastate an industrial organization and planning for continuity of operations under stress is vital. And so are such sound practices as regular backup and testing, air gapping whenever possible, and network segmentation. None of these, of course, are a panacea, but they're all solid practices. And Horizon3.ai's Principle Security SME, Stephen Gates, wrote that, "Attackers that gain access to any internal computing device are the primary threat industrial organizations face.” He also advises planning for continuity of operations, preparing for cyber events, just as one might plan for a fire or a natural disaster. The US Transportation Security Administration on July 26th released a memorandum announcing an update to its security directive regarding strengthening the cybersecurity of oil and natural gas. While earlier versions of the directive required oil and natural gas pipeline owners and operators to develop processes and cybersecurity implementation plans, the revision requires testing and evaluation of those plans. TSA administrator, David Pekoske stated, "TSA is committed to keeping the nation's transportation systems secure in this challenging cyber threat environment. This revised security directive sustains the strong cybersecurity measures already in place for the oil and natural gas pipeline industry.” The TSA website explains that every year, operators must submit an updated cybersecurity assessment plan to TSA for review and approval and report the results from previous year assessments. TSA requires 100% of an owner operator security measures to be assessed every three years. And operators must provide an assessment schedule that meets these criteria. Additionally, the update calls for operators to test at least two cybersecurity incident response plan objectives and include individuals serving in positions identified in the CIRP yearly. And finally, there's some bad news and some good news. Here's the bad news. Extremists are showing an interest expressed mainly online and in their social media jibber-jabber in using Flipper Zero devices to gain access to electrical distribution substations. The South Dakota Fusion Center, citing the New York City Police Department, warns that racially and ethnically motivated violent extremists, as they're characterized, have been chattering about the potential for using Flipper Zero to get into substations where they can wreak their havoc. Flipper Zero is described as an open source penetration testing tool for radio protocols, access control systems, and hardware. The NYPD has been seeing lots of videos in places like TikTok, where people purport to show how Flipper Zero can be used to open and close gates to get you access into all sorts of places you shouldn't have access to. Here's the good news. It's all on TikTok and extremists often don't know what they're talking about. As the Daily Dot puts it, "The Flipper Zero is a portable and digital multi-tool that can hack everything from radio protocols to access control systems. The device is capable of cloning RFID cards, such as those used to open hotel rooms, and has been shown to be able to bypass the security on certain brands of electronic safes. While the device is able to perform some impressive feats, its capabilities have also been greatly exaggerated in staged TikTok videos.” Flipper Devices, the people who manufacture Flipper Zero, which is a real item, also say it's a lot of hooey. It's been intentionally limited in what it can do, and among the things it can’t, is get through a modern access control system. So, while any industrial operations should certainly be paying attention to their access controls, it seems that Flipper Zero isn't exactly saying "open sesame" to your gates. But if you are vulnerable, that's what you should worry about, not the tool. So consider the minor furor over Flipper Zero an opportunity to look to your systems. Chris Wysopal, Founder and CTO at Veracode, offered a brief comment, stating, "Intelligence agencies are alerting on the possibility of white nationalists deploying the Flipper Zero hacking tool against the power grid. And its potential for bypassing access control systems.nI wouldn't call it a bypass. These systems are missing access controls and solely relying on security through obscurity. Organizations shouldn't fear the tool, they should fear the vulnerability.” And if you're an extremist who believes everything you see on TikTok, well you might have a big future in drop shipping. We're pretty sure we saw a video on that somewhere. It had a Lamborghini in it.

I recently had the pleasure of speaking with Mea Clift of Woddard & Curran. She shares her perspective on mentorship, internships, and apprenticeships, with an eye on OT security.

Mea Clift: So I started two days after I graduated high school as an intern with the National Cancer Institute doing desktop support. And that was back in the 90s, when security wasn't a thing. I mean, we were giving out public IP addresses to every system on our network. And that's like 2,000 machines at the time. And I look back now and I'm like oh my goodness, why did we do that? That's insane. But I worked my way up from there. I did some server support. Went into cloud, what was called application service providers at the time, before cloud became cloud. And then I worked for some other managed services companies. And worked my way up to the point that I had really reached the pinnacle and it was really either go into architecture or go into management. And I wasn't sure what I wanted to do. And so, a government contract came available, and I switched over. I thought I was going to be doing vulnerability management, they said no, you're going to be doing cybersecurity. And it was the best thing that could have ever happened. I jumped into doing documentation using NIST and GRC stuff. And that really took my career off. I was working on that kind of documentation for one contractor. They lost the contract, I went to another contracting firm, Bruce Allen. And then I moved over into their commercial space doing risk assessments for larger organizations. Which really helped me to see that I could analyze a situation using compliance to drive change in the organization. And so from there, I then moved into where I am today with Woodard & Curran, helping to build our maturity in our program and really elevate cybersecurity risk in the organization.

Dave Bittner: And what led you to your focus on OT security and specifically, in the water and utilities space?

Mea Clift: Honestly, working for Woodard & Curran has really kind of put me into the deep end on it. When I came on board and you know, I was learning about the environments, I'm like ooh, there's some big risks that we need to work on helping to mitigate and move forward. Not only to us as consultants but to the industry as well. And what's interesting is like my dad worked for Rockwell for a while right before he retired. He was a PLC, electrical engineer. And I was really kind of at the forefront in a way of seeing PLCs become networks because right before he retired, he called me today and was like how do I subnet? And I'm like -- why? I'm like I program your network, you shouldn't have to do -- and he's like well I'm working on this thing for work and I need to know how to subnet these networks all for this PLC stuff. And I said oh, there's a calculator for that. He's like you don't do the math? I'm like absolutely not, there's a calculator for that, let me send you the link. And he's like oh, this is brilliant. So, I got to see some of how that all interplayed before. Now he was working on sorters and manufacturing kinds of controls, not really in the water space. But when I came over here and I started to hear the same terms and started to see it, I said oh, I understand what you're talking about. And I was able to say okay, this is an area that definitely needs a little bit more focus and how can I help in that space to bring that focus in? And I have to give kudos to our engineer, Tim Maynard, he's been wonderful on educating me on the limitations and how our environments work and he's just a genius. I give him super props for dealing with me when I'm like hey, we should do this. And he's like yeah, we can't because of X, Y, Z. And I'm like oh. Well then, let's come up with some mitigations that we can do in this kind of environments. And it's been enlightening.

Dave Bittner: Well, but I think that really brings up a great point, which is the relationships you make along the way. And the ability to mentor each other. And fill in the knowledge gaps with your colleagues.

Mea Clift: I agree. Cybersecurity is ever-changing. It is ever-changing. We're having new things every day between new threats, new technologies, new capabilities, new theories, and you know, with AI now coming on board, it's a completely different environment. I remember back in the early days of my career there were always people who kind of held onto information thinking that being the keeper of the keys, per se, was their way of protecting their job and protecting their legacy. I've learned that I can't move forward if I'm holding onto things. And giving that information out helps to innovate. Helps to think in different ways. And then help somebody else move their career forward in a way that maybe they wouldn't expect. And I think it's kind of my duty as somebody who has stood on the shoulders of many people who have taken the time to teach me, to continue to teach others.

Dave Bittner: And how do you make sure that that information flows in both directions? You know, from the folks who are above you to you, and then you to the folks below you, and then I imagine there are things the folks below you tell you that the folks above you could make good use of as well.

Mea Clift: Well, I think that's kind of my responsibility in my role at Woodard & Curran. I'm a conduit of information in some ways. I'm learning from every aspect of the business, and I can. I'm reaching out and saying hey, you know, what's your business driver and how can security help? And raising awareness of key risks across the organization, no matter if it is a random pump station or a financial application that doesn't protect its data, I have to be able to say to everybody involved, hey, here's the concerns, here's the risk, here's what I think we can do about it. And then let the business make the determination. So, in some ways it's also a matter of educating. Sometimes folks don't even know that it's a possibility to do some of the things that we can do in cybersecurity. Or what butterfly effect for lack of a better term they can have to the industry as well. You know? One click of a phishing email can take down the entire network. Or greater, depending on how things are configured.

Dave Bittner: And what is the culture within the water and utilities space for information sharing? I mean there is an ISAC, right?

Mea Clift: There is. Water ISAC is phenomenal. It's run by Jennifer Lyn Walker, who is just amazing. Love her. And so, there's quarterly meetings, there's updates, there's regular training. They also have H2OSecCon in November that's two days of just presentations by cybersecurity leaders. They're taking presentation proposals and I'm hoping to put one in. And that gives you this community of professionals in the space and partners to say here's how we can help, here's what we're seeing, here's kind of the trending we're seeing and what you might want to do to protect yourself. And they also get information from CISA, CISA has done a ton around critical infrastructure and continues to, which is fantastic. You also have a lot of free resources in there, like CSET, which is a tool for compliance. You have IT Cert, which is partnered through Dragos, where you can get a censor to review. They also have Neighborhood Keeper, which also keeps an eye on the trends of what's happening in the industries in the spaces. So it's really great to have all of that information available to help these small municipalities do what they can with the resources available.

Dave Bittner: What's in it for you? I mean, it sounds to me like you do above and beyond when it comes to establishing these relationships and being very deliberate about the role you play in mentoring folks, probably outside of your technical job description. What do you get out of that?

Mea Clift: Honestly, just giving the next generation a chance. I do a lot of mentoring outside of my job through Cyversity, which is raising diversity and empowering underrepresented communities to get their cybersecurity careers off the ground. I also mentor through WSUS, and I mentor through ISACO which is another organization, a certification organization. I don't want to be doing this forever. But I also want to make sure that when I'm done doing what I do, I can drink my water safely. I can use the internet safely. I can entrust the next generation of cyber professionals to do what they're doing. So I can go off and do my hobbies or sleep until noon or, you know, do the retirement thing that I'm really hoping to do some day. And in order to do that, I have to give up my information, just like I was saying. I can't hold onto the keys to the castle. I'm not the only person in the world who needs this information. I'm not the only person who can do this. I want to make sure that everybody has all of the tools and resources that I can provide them. So that if tomorrow, I was hit by a bus, operations could still continue. And that the connections that I've built can cross connect and allow for better collaboration, innovation, and communication.

Dave Bittner: So for folks who are looking to take their place, you know, folks who may want to follow in your footsteps who find that your career journey is inspirational. Do you have any advice, any words of wisdom there?

Mea Clift: Definitely find your networks. Do the work of finding a mentor, working through things, finding your niche. There's so many broad niches in cybersecurity. You can really choose your own adventures. Or also, organizations I think as a whole need to start looking at the talent shortage that we have in cybersecurity less as a problem and more of an opportunity to grow incrementally. Our company does an internship program and we just hired our first cybersecurity intern this year. One of the advantages to doing that has been getting staff who come in and we're able to train them in the stuff that we work on, that we find important, and the projects that we're passionate about. So giving them that space and that opportunity and that knowledgebase allows them to potentially get hired on permanently here. And that helps the business. Because we get that retention going. We get a new, fresh set of eyes. And then we get to grow that person through their career. So it really is a benefit to them and to us. And we think apprenticeship and internship is really the future of cybersecurity, in order to get people in. Because there's not a lot of places to get that hands on experience unless you have that first job and entry level jobs are so hard to find in this space.

Dave Bittner: Yeah. You know, it really strikes me how important relationships are. I think I hear folks talk all the time about how it's hard to get your resume looked at. You know, there's so many gates and so many, even just like algorithmic gates. You know, if you don't have the perfect, you know, 10 years of experience for a field that's only been around for five years, nobody's going to look at you.

Mea Clift: Exactly. And I think there's definitely ways around that. You just have to show your deduction to learning your practices, whether it's, you know, playing with Hack the Box, or taking a class on JRC, or you know, building your own network in Amazon Cloud because you can do that for free to play around and learn their stuff, there's a lot of opportunity to point those things out. But I think it's also a responsibility of the business to start saying where can we lower the requirements here to bring somebody into the fold and train them up on how we're built and what we're doing?

Dave Bittner: That's Mea Clift from Woodard & Curran. [ Music ] Today's "Learning Lab" has the first part of a discussion about the convergence of OT and IT, with Dragos' Mark Urban and Kimberly Graham, Dragos' VP of Product Management. Here's their conversation. [ Music ]

Mark Urban: I'm Mark Urban with another episode of the "Learning Lab" on Control Loop. And today we're going to continue a discussion on convergence between OT and IT. When we had Rob Lee a couple episodes ago talk a little bit about convergence, he brought it into the SOC processes and we'll revisit that, click down on that a little bit. But then also get an understanding about how OT can integrate with other infrastructure pieces that are traditionally found on the IT side. And for that, I'm joined by Kimberly Graham, Dragos' Head of Product. Kim, welcome.

Kimberly Graham: Thank you.

Mark Urban: So first, let me recount at a high level what Rob talked to, and you know, he's been, you know, knee deep, neck deep into SOC processes for a good chunk of his career. But when we talked about OT and IT and where are convergence points, he brought it to SOC process. Really quickly, he was saying that level one, you want to consolidate all your alerts into a SIEM. You want a consolidated process, a converged process, so, you know, whether it comes from the OT side or the IT side, they go into one place, so they can be triaged and investigated at level one. If they merited, you know, graduation to level two, that's where you would pivot to an OT specific tool set to look at what was happening, right? Be able to investigate within the environment that the order came from. And then level three, when it progressed to that point, was about having an OT specific contact. At that specific site where the event was flagged from. And by the way, as someone who had been identified in the incident response plan, and hopefully that the SOC operators had met before, the investigators had met before through tabletop exercises. So, this harkens back to a lot of the processes we talked about incident response, borrowing on those best practices, and just kind of formulating them here. So Kim, if we look at that, you know, at level one consolidating into a SIEM, could you click down a little bit on what that means in OT specific alerts into, you know, a converged SIEM?

Kimberly Graham: Sure. So, you know, we have seen a lot of desire, which makes sense, to do some consolidation of where those alerts are received. Now the differences, of course, as you've already touched on a bit is that there's going to be a difference between an IT alert versus an OT alert. But when it comes down to it, in terms of that first line of support, it does make sense to have some consolidation there. And it is important to get any of your OT specific tooling forwarded into that SIEM. So that initial alerting, that initial triaging, can take place in a centralized SOC and just centralizing that as a process.

Mark Urban: Gotcha', gotcha'. So, a centralized process, I mean, we've seen, probably Splunk is the number one, you know, SIEM tool that I've seen in my career. There are many others. Once you move from level one to level two, pivoting to OT specific tooling, talk a little bit about OT specific tooling for monitoring the security applications.

Kimberly Graham: Sure, yeah. And you know, while you may be doing some consolidation at the SIEM to get that centralized process for the triaging, in terms of the actual monitoring that you're doing, it winds up looking quite a bit different on the OT side compared to the IT side. So if you look at things like the actual OT systems themselves, a lot of those OT systems, they're really originally designed just from a system and a critical standpoint, they're designed to be isolated and standalone. The network was more of a means to an end to get communication. And there's a lot of protonation of safety, operational efficiency, and up time. So there's not a lot of built-in security that you normally see in traditional IT systems, because they just have totally different threats. So you see these proprietary protocols and you see things that aren't patched, you know, very routinely. And so on. And you see a lot of that protonation of the general performance and operations over security. Because it's really reliability what matters. And you see that in the systems themselves, you see that in the networks that support the systems. So if you look at the threats that you see against OT versus IT, the threats tend to be more around the disrupting and damaging of potential physical processes more than you see things like data exfiltration, ransomware, and the things that you see in the IT space. Not that that doesn't exist, obviously we've seen ransomware can impact OT when you don't have the right kind of network segmentation. You don't have the right kinds of controls or monitoring in place. But in general, when you're talking about the types of threats that are specific to OT that don't overlap between the two, they're often very different in terms of the very nature of the threats. And of course the impacts are different. The impacts to IT can sometimes be up time, if it's like a DDoS. But a lot of times, it's data compromise and things like that, whereas again, the OT threats, the impact if there is potential down time, or physical damage of something, or even risk to human safety. So that changes the attackers' motivation. So you see, very different motivations between the actual threat actors between IT and OT. So it's important to have OT specific network monitoring that takes into account not only the way the systems communicate and the protocols that they use, but also the types of threats that you see in those environments. So when you look at writing detections, these aren't typically things like IDS detections. You can use an IDS to monitor a lot of IT things. And sometimes it makes sense to monitor those types of things in OT. But in general, a lot of your detections in OT tend to be behavioral. Is this a type of activity that we would expect to see? Is this going beyond what we would normally see in a normal operational environment? Or is this something that looks like a specific threat behavior? And that focus on behavior over just, you know, these raw signatures I think is a huge separation between IT tooling and OT tooling.

Mark Urban: Gotcha'. So there's the underlying industrial equipment that has a very unique environment of communications and kind of brittle in a lot of ways, if you know, memory service correctly. So in order to understand those very different systems and protocols, you need different types of monitoring equipment that specialize in OT. Is that a fair summary? Gotcha'. And okay, so, if level two is pivoting to an OT specific tooling to start to investigate, just talked about the importance of what are some of the characteristics of some of those OT specific tooling. But then lets go to a level three, right? Now an event is graduated, you know, and this is not a good graduation to level three, where site specific personnel now get involved. Can you talk about how, you know, how you see tooling impacting that kind of level three?

Kimberly Graham: So when it comes to level three, where you're actually engaging with the OT folks on site, whether that's the operations folks, the security folks, it's important that you have a tool that only does the monitoring that you want to see, but also provides information on, you know, what you can do. So there's various different levels of expertise that every organization's going to have at both the SOC and in the OT security space. And it's important to have tooling that can take you to that next step. This goes beyond a SIEM alert. This gets into things like playbooks. Playbooks can help you with how do I handle this specific type of incident? Is there any more information I should gather? And that can help to work with those, you know, on site people. The OT specific folks. To gather that data from an incident handling standpoint. So you can understand more about that. Or make sure they're not taking steps that would, you know, impede further investigation or something that you may need if there does need to be an escalation to a third party to continue handling that incident. Because you know, there's, Dragos of course offers services to do incident response. And you know, having that type of platform and that type of engagement, I think is key to actually being able to respond well to those types when they actually get to that level three.

Mark Urban: Gotcha'. So that level three has, you know, you do more investigation on site people. Starting root cause analysis. And you know, following those playbooks that can be prescriptive there. Also, and this is where it kind of transitions into an incident response plan, like if you find yourself at level three and continuing investigation, there's a point at which you'll activate your response routine. Or if you have that in place with a vendor, it's a point at which you'll start to go through your collection management framework to look at other sources of information, to do root cause analysis, and that you know, that gets at you're out of level three and you're into an incident response. So there's a -- there are a number of episodes that we have between Leslie Carhart and Vern McCandlish talking about in-depth kind of the incident response point. So I invite you to go back to those episodes to listen to what happens next. Because there's very, you know, good ways to approach that. Kim, thank you so much for your insights. And to everybody at Control Loop, thanks again for listening. [ Music ]

Dave Bittner: And that's Control Loop, brought to you by the CyberWire and powered by Dragos. For links to all of today's stories, check out our show notes at thecyberwire.com. Sound design for this show is done by Elliot Peltzman, with mixing by Tre Hester. Our senior producer is Jennifer Eiben, our Dragos producers are Joanne Rausch and Mark Urban. Our executive editor is Peter Kilpe, and I'm Dave Bittner. Thanks for listening, we'll see you back here next time. [ Music ]