CyberWire Live - Q1 2021 Cybersecurity Analyst Call

There is so much cyber news that, once in a while, all cybersecurity leaders and network defenders should stop, take a deep breath and consider exactly which developments were the most important. Join Rick Howard, the CyberWire’s Chief Analyst, and our team of experts for an insightful discussion about the events of the last 90 days that will materially impact your career, the organizations you’re responsible for, and the daily lives of people all over the world.

Transcript:

Rick Howard: Hello everyone. Welcome to the Cyberwire's quarterly analyst call. My name is Rick Howard, and I am the Cyberwire's chief security officer, chief analyst and senior fellow and yes that is a huge mouthful to say all in one sentence. I'm also the host of two Cyberwire podcasts. One, the CSO Perspectives podcast on the pro side, and one Word Notes on the ad supported side. But most importantly I'm the discussion leader for this program and I am joined by my long time friend and colleague, Bobbie Stempfley. She is the vice president and business unit security officer for Dell Technologies, but Bobbie, we've known each other for years right?

Bobbie Stempfley: We're not gonna mention how long.

Rick Howard: No, no like just yesterday right? [LAUGHS] So she and I worked together in the government, and she's also a regular on the Cyberwire's Hash Table. And last but not least, she's also one of the Cybersecurity Canoncommittee members because basically I twitched the arm of all my friends to help me with my pet projects. So thanks Bobbie for that. Joining her is Joe Carrigan. He is a senior security engineer at Johns Hopkins University's information security institute. He's also Dave Bittner's co-host of the Hacking Humans podcast. By the way, if you don't have that in your podcast rotating regime, you are totally missing out. So anyway, Bobbie, Joe, thank you for doing this and welcome to the show. You can say hello, go ahead.

Joe Carrigan: Oh hello. [LAUGHS]

Bobbie Stempfley: Glad to be here.

Joe Carrigan: Yes thank you for inviting us.

Rick Howard: So this is our fifth show in this series where we try to pick out the most interesting and important stories of the last 90 days and try to make sense of them. And this last quarter has been filled with all kinds of crazy nonsense. Just some of the stories we considered but didn't choose, one was the hacker that tried to poison a Florida city's water supply. Yikes! We had multiple adversary groups piling on the Microsoft exchange vulnerability. Some ten groups in all. Six of which began exploiting it before there was even a patch. That was scary. And we had in terms of physical security, here in Washington DC, that disgraceful Capitol riot. So all of that going on. But Bobbie, let's start with you. What's the topic you chose as being the most important for this past quarter?

Bobbie Stempfley: Yeah it's really interesting, you're right, the choices were really rich [LAUGHS] for us to pick from. I actually chose the power issue in Texas.

Rick Howard: That was a unique choice. I didn't expect it. That's a really good one though.

Bobbie Stempfley: Yeah. I went back and forth. It is not mainstream in the cyber vein, but I look at how much all of these stories have a common thread of systemic risk issues, and enterprise understanding, and third party concerns. And on this path of digital transformation and I think about what CSOs are facing right now. If you believe the metrics, 75 percent of all data is being moved to process at the edge by 2023. 75 percent of all software is being bought in a consumption model, and something like 50 percent of all Fortune 100s are turning to robotic process automation. So this confluence of all these activities means we really need to understand these enterprise systemic risk questions and with increased complexity at an incredible rate, and complexity is the enemy of security and resilience. That was the lesson for me out of the Texas issue.

Rick Howard: I can confirm what you're saying .The Cyberwire's basically a start up. We have some 20, 25 SaaS applications that run the business. We have nothing back in the traditional headquarters, so talk about the edge. But Joe, help us out here. Can you describe what the edge means, when she says 75 percent of the data process is at the edge? What does that mean?

Joe Carrigan: Actually, that's an excellent question. I was going to ask for clarification on that. [LAUGHS]

Rick Howard: Oh well you and I both don't know what that is. So Bobbie, tell us what that means.

Bobbie Stempfley: So think about it, we currently operate in this model where the data is moved to a central cloud for automation and processing. So much of it is being collected in the sensors, out at the edge, and the processing is being pushed out to that edge for processing and execution. So think about it in terms of our IT and OT spaces. Those devices that are collecting and sensing are now going to need the analytics and the orchestration to happen in telco locations, in factories, in other edge organizations, and that is the future.

Rick Howard: Go ahead Joe.

Joe Carrigan: I was going to say that one of the driving factors behind that is the absolute reduction in cost of these units that are capable of doing processing. I have right here an ESP 86 or something, but this is a dual core processor. It cost five bucks. And it can run a web server on it.

Bobbie Stempfley: Right. So processing is lower, communications is better, and there's no reason to move the data around in order to have the analytics that you need. And that's enabling a phenomenal amount of innovation there at the edge.

Rick Howard: I think it also means, to clarify it a bit more, it also means that the processing is being done right where the internet connection is. In the old days when I was a youngster, we'd have these big data centers and you'd have to back haul traffic all the way back to headquarters and then it would go out to the internet pipe and that's all bandwidth intensive and hard to manage and very expensive. What I think you're saying Bobbie is that you put it out at the edge, and you process it and get it out and there's no time wasting there.

Bobbie Stempfley: That's right. You reduce latency, you increase productivity and you add to it this challenge that we're seeing, which I think we also learned from the Texas event, even more acutely, the IT infrastructures and the OT infrastructures are being even more tightly integrated. Now we have a whole different set of security challenges to think about.

Rick Howard: So for our audience that weren't familiar with what happened in Texas, maybe we should just pause for a second and explain what happened. So Bobbie tell us what happened.

Bobbie Stempfley: Texas experienced a phenomenal weather event and that event...

Rick Howard: I will say [LAUGHS].

Bobbie Stempfley: ...yes it snowed. And that weather event caused major pressure on the electric grid and due to a series of technical and policy issues, they went through a series of rolling brownouts and blackouts, and people were in a really difficult situation for a period of time, about a week. It caused outages in data centers, in infrastructure locations because of the power issues that were occurring.

Rick Howard: What comes to mind too is most people when we think about big black swan events like that, and big outages, that we're worried about being able to process our data and still deliver our services. But I think there's another piece of resilience that most of us don't talk about all the time, and it's mostly what I think about in terms of people and process. So is that a part of the resilience strategy for organizations? Did you notice Bobbie as you talked to different customers out there?

Bobbie Stempfley: Yes. So I think you bring up an incredibly good point and actually it's one of those where I love the way Dell approached it was really very much a team member first model for Dell as a company. But when you think about organizational and enterprise resilience, you have to plan for the human capital resources and the technical capital resources that are there. I think you point out the fact that security officers need to include availability of those resources in their enterprise resilience strategies. We often times think about adding an insider risk approach, thinking about people as a risk we have to understand and manage. But we have to think about them most importantly as an asset that we need in order to plan for these kinds of events.

Rick Howard: We have a poll question for that, Kelsey wants you to throw the first poll question up on the screen, and all the audience members, you guys can click your answer there, to see what you think. See if anybody in our audience is planning resilience strategies around people like Bobbie was talking about. And Bobbie, Dell they have a big presence in Texas, so it's one of the reasons it's on your mind, right?

Bobbie Stempfley: Yes, yes absolutely. The headquarters is right there and it affected a large number of team members in Dell's world. But you think about the kind of challenges that all companies are having managing any number of these large scale systemic events. Over the course of the last year, we've had any number of them.

Rick Howard: So, that's not bad, 25 percent say they're thinking about it. But 75 percent says no. That's interesting. I don't know Joe if you have any comment about any of that?

Joe Carrigan: Actually I was thinking about one of the things that Bobbie said with regard to complexity being the enemy of security and if I can go back to that, that was what I thought was what interested me. Every single time you have some new system talk to another new system or existing systems, when you build this complexity you're building another interface. And every single interface should be evaluated in terms of its security and I would hazard a guess, I'm not a big gambler, but if I was a gambler, then I would say that the vast majority of these interfaces that are put out there in the name of getting things done quickly and bringing things to market, the vast majority of them have not been tested for security. Or even designed for it.

Rick Howard: I would also throw out there that I don't think that many CISOs or CSOs, you take your pick even consider that part of resiliency or of complexity in terms of where people are to get the job done. I don't think that even falls under their bailiwick,so I don't know who's even looking for it in many organizations. 

Joe Carrigan: It's a system engineering problem.

Bobbie Stempfley: Yeah. But back to, the reason that I picked this story is that it was endemic of I think all of the ones we could have chosen. It really identified issues that were common across them. Understanding your connections, your third party risks, your supply chains I know lots to talk about there, the people component to it. As a chief security officer, how you think about a converged security organization where you can take advantage of what you might know from a physical security side in your cyber environment and vice versa. It just really brought all of those issues to the forefront.

Rick Howard: So while we're talking here for audience members, you guys can put your questions in the little question pad there and we'll try to answer them as we go. Bobbie, we've got one here from Tickle Me Elmo which is probably the best username on the planet. Here's what he or she says. " Akamai Technologies operates one of the world's largest content delivery networks. And they keep most of their computing capacity in six data centers that were affected by the storm, so as a potential customer of Akamai, what should I..?" This is Tickle Me Elmo talking, what should Tickle Me Elmo be thinking before they buy their services? What questions should they be thinking about?

Bobbie Stempfley: Yes, I think what this really highlights are questions you need to ask all of your suppliers and it's about their resiliency programs. What are their projects and focus on bringing their services back up and sustaining their service in large scale events that are there? And this applies not just to Akamai or content delivery related activities. Think about the rash of ransomware events that are going on right now. How would you work with somebody, how could they be restored from one of those events sufficient to not impact your business?

Rick Howard: So Joe, ransomware attacks are in your bailiwick right? So what kind of advice do you give our listeners about resiliency in terms of that kind of thing? What should we be planning for?

Joe Carrigan: You should be planning for someone to encrypt all of your data. And steal it. That's the other part of it. So really there's two things you can do to protect yourself. Number one is encrypt your data at rest, or even if you can, when it's in use and only decrypt it when you need it. That prevents these malicious actors from getting your data and disclosing it or selling it which is what we're seeing happen more and more in ransomware attacks. And the other one is, of course, what we've been saying for years about ransomware is back up your data and make sure those backups are tested. You know, if you can't demonstrate that you can restore that data from a bare metal system, then I would argue that you don't know that you have good backups, which means you may have a false sense of security.

Joe Carrigan: People have gotten better at that over the past couple of years, and I think that's what has pushed these malicious actors into taking the data as well. So to protect yourself there, if you can, encrypt the data when it's at rest, that way when they say, "we'll release your data," you go, "it's encrypted and you didn't get the keys. So go ahead and release it, it's fine."

Bobbie Stempfley: Yes, but I think that testing the backup is very much a resilience thing to do. So understand what's most important to you and test those processors.

Joe Carrigan: Yeah, test the processors, do table top exercises for your business continuity plan. Those kind of things. One of the great things I heard recently, we talked about on one of the podcasts, it came over the Cyberwire Daily or the Hacking Humans podcast, but in an executive meeting if you're the CEO of a company, just grab a newspaper and read what's happened to some other company and ask your team, "What would we do if this happened to us?" It's a great thought exercise and it kind of gets people into the process of thinking. People should have good answers to that, and if they don't have good answers to that, they should start thinking of good answers to these questions.

Rick Howard: Well Bobbie resilience has been popping around the security field for many, many years and people have lots of definitions for it. You want to take a swing at defining it for everybody here? What does resiliency mean to you as you talk to your customers?

Bobbie Stempfley: Resiliency means the ability to work through and come back from an event with minimal loss of business mission impact. That's a pedestrian definition, you can decompose that into a lot of different ways. But I think it's really important to have a definition that even my mom understands because when you end up with the more difficult detailed ones, you actually end up making the problem less understandable. And therefore less solvable.

Rick Howard: I've always understood it that way too that you can have a major outage internally but because you designed your system so resiliently, that none of your customers notice. So that'd be one way to think about it. We got a question here from username, Your Name Here, which I love that name too. To your point he says, "Resiliency means a lot of things to a lot of people. What would you say it means in terms of business and security operations?" I think what the user's asking here is it's not just having technology that makes your system resilient, but your people and process.

Bobbie Stempfley: That's right.

Rick Howard: So how do we think about that?

Bobbie Stempfley: I think you actually start there. So I think Joe's comment, you have to start with what's most important. What's the job that the business has to be wholly focused on getting and accomplishing in these events? And then how do you decompose that into the other processes, the people and the systems that support those kinds of efforts? It might not be the most clear cut question when you're thinking about operating under one of these really significant events. It might be that the thing that's most important is not the same thing that's most important in your everyday activity so you really have to think through both of those. And I think Joe your question about looking at the newspaper as a CEO and saying "How could I keep this from happening to me?" I think the same question has to happen in every CSO and CISO's daily news review and every board members as well.

Rick Howard: So that's some excellent stuff Bobbie so thanks for bringing that topic to the table. It gives us a fresh air perspective on something that we don't normally think of. But it's time to shift gears here and go over to Joe's topic, so Joe what was your big thing for this talk? What did you think was the most impactful story in the last 90 days?

Joe Carrigan: The one I thought was the story that broke back in January. It was a Google story from Google research. I can't remember the author's name, I talked about it on the show. Adam Wiederman was his name, or was it?

Rick Howard: I was going to say Joe, that's the first sign OK? That's the first sign that you start losing.[LAUGHS].

Joe Carrigan: OK then I'll look for other signs as well, I'm sure they're there. There was a pretty big social engineering angle on this story. These malicious actors were setting up Twitter profiles that they used to post links to their blogs. They would amplify and retweet each other's tweets from these fake accounts. They would post videos, and at least one of these videos was faked of their claimed exploits, and then what they were trying to do was engage with security researchers who found vulnerabilities in software and applications and operating systems, and they'd say "Hey, do you want to collaborate? Because I'm working on this vulnerability here," and they would send over a visual studio project with a malicious DLL in it. And, I thought that was a great way to deliver something to somebody. You're somebody who works on malicious stuff, you work with a tool like Visual Studio which develops DLLs that may require permissions. But what wound up happening was this DLL would install a back door on their system, on these researcher's systems. Additionally, the blog was malicious as well. 

Joe Carrigan: So when they posted these links if you were to go to these, they were exploiting zero day vulnerabilities in Chrome and in Edge and in Explorer. That was one of the main points of the article, was that this organization had spent zero days to try to collect more zero days and they wanted to know how they were being exploited. Some researchers did realize they were being targeted. Go ahead Rick.

Rick Howard: What's the economics for that? Of costs, just general ballpark, zero day cost, a million dollars to develop, so you're going to blow that one to get your line on another one? Is that what you're saying?

Joe Carrigan: To get hopefully several more. I mean you want return on investment.

Rick Howard: That's what I've heard.

Joe Carrigan: Turn a dollar into a dollar, you want to turn a dollar into ten dollars. So the same is probably true with these other tools. These people probably developed these or found these exploits in these browsers by themselves and they were using those. Like I said, those systems were fully patched. So these were up to date systems so they were new vulnerabilities, and exactly Rick, your point is if this vulnerability is worth a million dollars and I can spend it or maybe it's not worth a million dollars, maybe it's work $100,000. And I can invest that to get a foothold into a researcher's computer and get what would be a million dollar vulnerability, and I can sell that to somebody who buys them, when the researcher's plan may have been to ethically disclose it to the manufacturer or to the responsible individual like we do at Johns Hopkins where I'm actually responsible for disclosing these vulnerabilities to people, and I have to chase people down and tell them we're going to disclose the vulnerability if they don't fix it.

Joe Carrigan: It's really hard to do if the company doesn't have a vulnerability disclosure process involved and a lot of time you don't get any feedback. I think they were either trying to monetize it because these things do have value, or they were just trying to get more access to more systems.

Rick Howard: Go ahead Bobbie.

Bobbie Stempfley: It's a really interesting way to go about it, right. What is the saying? "You rob banks because that's where the money is."

Joe Carrigan: Right [LAUGHS].

Rick Howard: Exactly.

Joe Carrigan: You attack security researchers because that's where the vulnerabilities are.

Rick Howard: It kind of goes to the point where a lot of people, and Bobbie I'm going to come to you with this because you probably saw a lot of this when you were working for the government, right. But many people said "What am I worried about hackers for, I don't have anything they want?" And here's a whole new pond of research that is being extracted from people who weren't typically targeted before.

Joe Carrigan: Yes, I actually have a number of slides on that topic alone. Yes, you are of interest to hackers. Bobbie you were going to say something?

Bobbie Stempfley: Yes, well the interesting thing is over the last ten years, to your point about coordinated vulnerability disclosure. There's been a pretty big push to try to grow companies' awareness of their responsibilities here and maturity in that activity. And even international standards around what coordinated vulnerability disclosure is and how it should function. And so trying to draw the researcher community as meaningful constituents and partners, there's a very interesting social dynamic to this Joe that I really find. I can see why you found this as the most important issue.

Joe Carrigan: Right. A lot of times what has happened in the past, I don't know how often it happens any more, but when a researcher finds a vulnerability in some product they'll disclose it and the company will go "That's great. If you disclose this publicly we'll sue you." Which is not the right answer, it's not the right course of action. Volkswagen did that with one of their keyless entry systems. And the researchers were actually beaten into submission for a number of years, until somebody started exploiting the vulnerability.

Bobbie Stempfley: Yeah, and that's not responsible.

Joe Carrigan: Right. That's absolutely not responsible.

Bobbie Stempfley: The coordinated vulnerability yeah, agreed.

Rick Howard: This is kind of an adjacent idea but the way that these hackers attack these researchers, it's kind of a version of a supply chain attack, right? I mean you could look at it that way, right?

Joe Carrigan: Yeah, I would look it more like the mob hijacking a shipment of cigarettes. Like these cigarettes are made for one purpose, to sell legitimately. I'm going to go and steal this. I guess you're attacking a supply chain but it's not like the SolarWinds supply chain attack or other supply chain attacks where you're intercepting something. You're going further up the supply chain of vulnerabilities actually. And after my poll question I have some comments on that and the life cycle of a vulnerability.

Bobbie Stempfley: Aren't you attacking the supply chain of the researcher?

Joe Carrigan: Sure.

Rick Howard: That was kind of my point. And the software that they used, they used Visual Studio, right. So Kelsey brought Joe's poll question up, OK. So they had exploits for Visual Studio that nobody knew about.

Joe Carrigan: Well no that wasn't an exploit, they just wrote something malicious using Visual Studio as a tool. And the idea was, it was more of a Trojan horse situation than an exploit.

Rick Howard: So they wrote a piece of malicious code inside a Visual Studio that wasn't an exploit but it did bad things once the victim ran it.

Joe Carrigan: Right exactly. It installed a back door onto their system.

Rick Howard: Let's see the results here. Most people are allowing automatic updates and I guess the question is, is that good or bad? Joe what do you think?

Joe Carrigan: I think that's good, that most people are doing that. I specifically couched this as an end user system, not as a host for your services and systems in your server room and all that. Because generally speaking, you don't have a lot of configuration management with most end users. You run an operating system update, you're not too concerned if the software that they use is not going to stop working which may not be the case with a server product. In my early days I was a system administrator for a document management system and we would absolutely not update the database on that because the document management system didn't support the next level of database. Maybe it would work just fine, but because it wasn't supported, we wouldn't do it. So I understand why people don't do that. But on the end user side, these updates should be pushed out.

Joe Carrigan: So in February what happened, Google had a patch for Chrome that was linked to this zero day and in March Microsoft pushed out patches for Explorer and Edge. I want to talk about the life cycle of one of these vulnerabilities. First off, think about it, there is a vulnerability in some software, it's out there all the time. But if nobody knows about it, the risk is very, very low. It's not nothing because you could accidentally do something to yourself, but it's very low. Then once somebody discovers it, I say the risk is still very low unless you're one of the people that's targeted. So the chances of you being targeted by somebody with a zero day, like we've said, these are expensive things. They're very valuable, so unless you're somebody that knows you're going to be targeted, or thinks you're going to be targeted, or in the case of these researchers, maybe they didn't know, you're probably not likely to know about it.

Joe Carrigan: Once the vulnerability is disclosed to the manufacturer and assuming an ethical process, the coordinated vulnerability disclosure, your risk is moderately higher just because more people know about it. But when your risk really increases in this, and this is not something I've heard anybody else saying, but your risk is really increased when the patch is released. Because if I'm a malicious actor, and I see that Microsoft has patched some vulnerabilities, all I have to do is have a system, see what gets updated, then go to an older system, look at the code that got updated and see what the vulnerability is. I can do a reverse engineering task on that and relatively easily find out what the vulnerability is. So essentially patching is almost a disclosure. It's a pretty advanced disclosure or for more advanced adversaries it's effectively disclosure. that's when your risk is only going to go up. So once it's been patched, the next step is somebody's going to talk about it and say here's a proof of concept, we're going to put it into Metasploit and other vulnerability scanners so we can find it.

Joe Carrigan: Public disclosure and then incorporation in to tools, things like Metasploit, that's when your risk is only going to go up. So once it's been patched, the next step is somebody's going to talk about it and say here's a proof of concept, we're going to put it into Metasploit and other vulnerability scanners so we can find it. But once that's out there it becomes more and more available to the more and more less skilled attackers. So my key takeaway here is that once a patch is released, that's when you should understand that your risk is really increased a lot and that's why you should apply these patches as soon as possible.

Bobbie Stempfley: We keep doing this Rick, I'm sorry. You'd think we'd have gotten this worked out over the last couple of years. The New York Cyber Task force I think four years ago did a study, and as a part of it they showed that auto updates was one of the most significant risk reduction activities that could be taken. So I was really pleased to see that 75 percent figure in the poll, and I think to your point, auto updating patching becomes really important.

Rick Howard: I want to play devil's advocate there right, because you just talked about the SolarWinds attacks, the auto update mechanism is how the bad guys got in, because they compromised...

Joe Carrigan: You're absolutely correct. That's one of those supply chain attacks. And people were doing the auto updates and doing what they should have been doing. They attacked the best practice. The SolarWinds attack is remarkable in how effective it was. I don't know, do you think we're going to see more of these in the future? Probably. I think somebody exposed it as a real vector. It was remarkably successful so I think we're only going to see more of it. That's the way the economics of this situation works.

Rick Howard: So Joe we got a question from a user called Mouse Rat Rock Band, which is awesome. He says "Can you describe the attack sequence the hackers use to compromise their victims?" In other words, what was the intrusion kill chain they used? Can you give us a run down on some of that?

Joe Carrigan: Yes, well it was really a multi pronged attack using social engineering to engage with these researchers. Then something as simple as sending a malicious Visual Studio file or tweeting links, and retweeting links to malicious blogs that had effective zero day tools on them, exploited zero day vulnerabilities.

Rick Howard: One of the things I liked reading this story was the recon was extensive. They used lots of social media channels to find some of their potential victims. They used Twitter, they used LinkedIn, they used Telegram, they used Discord. Key Base, I didn't even know what Key Base was, and they used email to track down these vulnerability researchers and get in good with them so they can exchange some things with them, right. And convince them to join a Visual Studio project, and that's where they got had. I kind of liked that idea.

Rick Howard: The other thing I liked about it too is they did two kinds of delivery. For the ones that they couldn't get to use the Visual Studio thing, they actually did a watering hole attack going after Internet Explorer users. So this was not a fly by night thing. This was kind of a thought out attack sequence that these guys did. So pretty interesting.

Rick Howard: Bobbie, let's talk about that SolarWinds thing again because having someone deliver code through a back door, that might not be the most risky thing,right, it's just code. What we really want to do is stop them, after that happens from gaining access to some really important pieces of data. So that's where the control mechanism should come in place, right?

Bobbie Stempfley: Well I think you have to have controls in a variety of places, one of which is obviously verification of the code as you're accepting it. The intricacies of the SolarWinds attack really covered several steps that we've taught people to pay attention to. But yes, we need to have controls at the execution point, and I think that two things that are takeaways for me from the whole conversation, is back to the people component to it. We need to really understand the motivations of people and how we keep them focused on the things that are important. The business and security operations that are important parts of our sense of responsibilities.

Bobbie Stempfley: Then the second one here is really understanding your business processes and interface points in all those business processes, and so where you would put a control in place to keep the code from executing and particularly in an unusual way or recognizing as early as that business process went awry, is an important part of how I think about organizational and business resilience.

Rick Howard: Joe, we got a question from Got a Segway, and his question was, "Did Google attribute the attacks to any particular group?"

Joe Carrigan: The attacks have been attributed to North Korean operatives. I think Microsoft said it was the Lazarus group. I don't know. Attribution is notoriously hard. But I guess there's enough information that people feel comfortable saying this was definitely North Korea. I don't know that I feel comfortable saying that, but it wouldn't surprise me if this was North Korea. It doesn't seem like it's something out of character.

Rick Howard: Yes, I think Google said a non-attributable North Korean group, Microsoft was more specific. They called it their Zinc group which you said, which everybody knows is Lazarus which is by the way officially the military reconnaissance general bureau of North Korea. But attribution like you said is really hard, so maybe not. [LAUGHS] We're going to cut it off there Joe. That was a good one, really interesting topic and out of the ordinary so I love it. Thank you for that.

Rick Howard: We're going to move over to my topic, and we've been kind of bouncing around this whole idea of supply chains and thinking about that because even our show for last December we were all watching in earnest as the first news hit the wire of SolarWinds, right. And it felt like our colleagues in the security community were just realizing that supply chain attacks were kind of a thing. I know that's not true but with all the press coverage, that's what it felt like. Are you guys fans of Casablanca? It's one of my favorite movies. You know the scene when Captain Renault, he's going to close down Rick's Cafe for no real reason. He uses the fact that he's discovered gambling in the back room and he tells Rick, "I'm shocked, I'm shocked I say to find gambling going on" and the cashier walks out and hands him a pile of cash, he says "Your winnings sir." I love that movie. [LAUGHS]

Rick Howard: That's what I felt like for the SolarWinds attacks and supply chains in general. We all kind of felt like Captain Renault, we didn't know they were actually out there. Even though there's been a slew of high profile supply chain attacks in the past and Catherine Stevenson pointed out in the chat room, SolarWinds are not [UNSURE OF WORD], OK, have been just in the news of recent years.

Rick Howard: So we started going back and looking at all the supply chain attacks, just this last quarter. Let me just run down a few of them to show you how robust this is. The FIN11 cybercrime group, they went after Acellion'slegacy file transfer applications and they extorted over 100 different companies. We got research from a guy by the name of Alex Berenson,I think is how you say his name? He published his research on dependency confusion or name space confusion. He discovered malicious code in all kinds of public software repositories like NPM and Rubygems. We got ESET reporting on operation night scout campaign. Those guys infiltrated a software package called NoxPlayerthat emulated android operating systems on PCs and Macs and was used by Android gamers. RSA published research on the Kingslayer campaign, another fantastic game. They infiltrated the company's software update system, similar to SolarWinds. This is my favorite one, the New York Times said a Czech company called JetBrains, have you guys heard of JetBrains before? I'd never heard of them before.

Joe Carrigan: They make a typed on interface, I have it on my computer.

Rick Howard: And they sell to 79 of the Fortune 100 companies, right. And they said the Russians compromised their team's city software. Unbelievable. And the last one I have is the French security agency Ansesannounced that Sandworm, another instance of Sandworm, exploited an IT monitoring tool called Centreon. So here's my question to you guys, right. In all of our organizations, people are in charge of security, but who owns the supply chain in most organizations? Does that fall under the typical CISO or CSO role or does somebody else have that responsibility?

Bobbie Stempfley: I think we have to be really careful about how we define supply chain back to our earlier comment. In Dell, the digital supply chain is owned by the Chief Security Officer. We have a series of programs around it. But the physical supply chain like the actual moving of hardware and software components, is owned by a different organization. The two come together and you have to partner in that activity, but I think back to our earlier comment about being shocked that there's gambling in the organization. It's a thing that organizations have to be clear about. Assign that responsibility and accountability.

Rick Howard: So Kelsey let's put up my poll question and we'll ask the audience how they do it. Joe, what's your experience? As you're out and about talking to people, who owns the supply chain problem in most organizations?

Joe Carrigan: I don't know who does that and most organizations actually don't have much time to talk to people about that. Or much experience talking to people like that. But I like what Bobbie said about it, that you have to be careful how you define it. It's something that you have to think about and the security of these different parts of the supply chain are of paramount importance as we've seen with, well let's say again the SolarWinds attack. Who is responsible for making sure that the product that you're downloading from your supplier is safe and secure and doesn't do anything it's not supposed to, like for example phone home? How do you check for that? Who is responsible for that update? Or for managing that update and making sure that it maybe exists in a sand box for a little while.

Joe Carrigan: I don't know where that would be. I mean it seems like it would be something in the CISO organization, but when you start talking about supply chain you might also not be talking about things like infrastructure maintenance. You might actually be talking about raw materials. I'm reminded of my grandfather worked for a paint company. He would buy raw materials by the paint load, that's part of the supply chain. There are probably security issues in that as well. Who's responsible for that kind of thing?

Rick Howard: Let's see what we have here. That's interesting [LAUGHS].

Joe Carrigan: Yeah a lot of people say somebody else.

Bobbie Stempfley: That is.

Rick Howard: It kind of scares me that 90 percent say what's a supply chain? So that's kind of where we are in the industry, right. The interesting thing, Bobbie you and I have had to deal with this in the government in dealing with financial organizations, they've been worried about supply chain for a long time. And one of their tools that they use all the time was the risk assessment questionnaire. Some of these big financials would, you know, these long questionnaires that they would make their suppliers go through. Tell me how effective those things were?

Bobbie Stempfley: Yeah, so I think they're effective in two ways. One, they're a great signal to the supplier of what's important, and what needs to happen. So one of the things that I like as sort of an industry trend, is that there are more and more of these pre-engagement questions. So I'm going to reflect back on the question that I think it was Tickle Me Elmo asked about Akamai. That's a pre-engagement question that it really is, help me understand what my supply chain risk is because you're a supplier to me. So I really like that trend that's there.

Bobbie Stempfley: The other place that I think they've been helpful is particularly for organizations that don't have a good security program. There was also a follow up dialog in many instances between the bank and this or in the defense department, the large suppliers with the smaller suppliers about growing the maturity of their programs that are there. I'm a believer that you have to have technical controls and procedural controls that questions are you answering the questionnaire yes or no is interesting, being able to describe whether you have technical controls that really implement each of these becomes even more important.

Rick Howard: I was reading some websites before the show started, and KnowBe4 did some research and showed that mosts of the attacks coming out of cyberspace are coming from third party attackers, third party supply chain. The number they quoted was like 74 percent of them, so it's not a one off like we all thought SolarWinds was. It's more of the norm. We got this question from Jeff Alder,Bobbie and it mentioned something you said before, so I thought you might want to convince on this, right. This is what Jeff says. "Will SolarWinds push a zero trust approach to cyber security even further? And to assume a breach and then maybe use more artificial intelligence somehow to change it or something?" I don't know, what do you think about that?

Bobbie Stempfley: I think that we are at a point where the environment itself is producing an amazing amount of data about the environment. And if we are smart we will use the kinds of technologies like a variety of different AI capabilities in order to process all of that. And we'll use this to be transparent where we can in different areas. So I'm also a big fan of software bill of materials, so that suppliers can be pretty clear about what's in their offering and transparent about what products they are building on and what they're using. That helps with our vulnerability disclosure activity, because it helps you understand when you're vulnerable to those pieces. And the automation of the whole development tool chain gives us a chance to now really do this in a meaningful way.

Rick Howard: Well this goes back to something we were talking about in a previous section that the actual back door, and let's take SolarWinds, wasn't the big issue. The back door didn't cause the damage. The damage was caused after the bad guy got on the end point, was able to create authorization tokens for cloud resources. So to me the thing that stops that is a zero trust approach about who has authorization to authorize tokens for access to the cloud. That's where you stop that attack, not with the back door, right.

Bobbie Stempfley: It's back to our past, you can't just solve this problem in one place. You have to have controls at every layer.

Rick Howard: At every layer yeah.

Joe Carrigan: Right, and as a purchaser of these kind of products, these questionnaires are great, but how do you validate that your suppliers are actually doing what they're doing?

Rick Howard: Well that was kind of my point too that a questionnaire is one thing, and like Bobbie said, it puts the supplier on notice that you're worried about it. But how do you verify? That's a whole other thing, right.

Joe Carrigan: Yeah.

Bobbie Stempfley: Yeah.

Rick Howard: Joe, we've got a question from Kevin, back to a previous section. You mentioned interfaces. He wants to know how you define interface?

Joe Carrigan: I would define an interface as any point where two systems talk to each other. Say for example, an embedded system, you might be looking at how the sensor talks to the processor on that embedded system's device. But there's also an interface that that sub-system uses to communicate back to maybe a server where it's collecting events. Every single one of those things is an interface and you need to worry about that. In configuration management and in systems engineering.

Rick Howard: We got a question from Granny for the Win, fantastic name. She says, "Besides software update services, like SolarWinds, what other kinds of supply chain security issues should we be thinking about?" You guys got any thoughts about that?

Bobbie Stempfley: Well I think that this is another place where you need really a third party risk management program. Your supply chain is all of your suppliers, and understanding who those are and what kind of controls they have in place is important. Back to the earlier comment about complexity and Rick, your experience at Cyberwire, all of the different as a service, offer capabilities that are being leveraged, understanding what risks those present and what benefit they present is I think an important obligation that Chief Security Officers have.

Rick Howard: I'll give you another one too. I'll flip it on its head. It's one thing to worry about supply updates, software updates from SolarWinds and anybody else for that matter, right. But also if you're a big enough organization, you're probably like Dell let's say, you're operating all over the world, and you have people in your organization choosing where they're going to put office space. In my past experience we had some of our finance folks looking to establish offices in countries that maybe we shouldn't be there because of the way they operate. I'm talking about maybe you don't want to open up an office in say, China or Russia or somewhere in Eastern Europe because of the way they treat the data in those countries. So that's another way you could look at supply chain, is where you're going to operate from because it might be cheaper to operate there, but maybe the risk of compromise is greater too. So, it's something to think about, OK. Bobbie, what do you think about that? Am I off base there, or?

Bobbie Stempfley: I think that understanding that we have to source talent from wherever, there is a talent shortage and sourcing talent is an important business decision that's there. I think every company goes through that risk calculus. What's the legal framework that exists there? What's the employment framework that exists there? What's the risk to IP? How do you think about all of those in making decisions about where to put different processes. Particularly in this global market I think it's very real for us to have to consider.

Rick Howard: We've got a question from Wayne Lloyd and he says "Do you see the industry planning for being CMMC compliant in order to do business with the DOD?" And he says "CMMC addresses supply chain risk for the DOD." Is that something on the horizon for us?

Bobbie Stempfley: I'll let you start Joe.

Joe Carrigan: I think it's coming. I think that the CMMC is, that the DOD's going to start requiring certain levels of CMMC compliance. Yes. I can absolutely see that.

Bobbie Stempfley: CMMC is the cybersecurity maturity certification. It's intended to be an approach to ensuring that the defense industrial base reach is at a level of risk appropriate for the defense department and appropriate for the kind of capability that that particular provider provides. Understanding that you know, a Lockheed Martinwould be different than somebody that provides pencils into the defense department. And yes, absolutely I agree that CMMC is, the [UNSURE OF WORD] has been pretty clear, CMMC is on its way. The CMMC accreditation for it just appointed a new director and I think the industry is preparing.

Rick Howard: We got a question from Warning Low Battery, again his point was that he's not sure where all those applications are, and so how do you go about finding your third party applications if you're not familiar with them? What should a CISO or CIO do to do that kind of thing?

Bobbie Stempfley: Start with your procurement organization.

Rick Howard: Oh.

Bobbie Stempfley: They're paying somebody.

Rick Howard: That's right. Go ahead Joe.

Joe Carrigan: And one of the winners of best product I think at either Defcon or Black Cat a couple of years ago was inventory management product. They'd just simplified inventory management, and that's what this problem is. You don't know what you have or where you have it, and building on that you don't know what other people have put on their systems to do their jobs. Maybe you have somebody who uses an open source piece of software that helps them do something and that piece of software comes up having a vulnerability in it that's exploitable remotely and you don't even know it's on your network.

Rick Howard: Well I'd offer two suggestions. One you could probably do it with tools that you already have. Just to get a kind of rough cut of it. Most of us by this point are running some sort of next generation firewall, that's a layer seven firewall. That means they are identifying applications running on your network, so I would start there, OK. Just get a listing of all the applications running through your next gen firewall. The second thing I would offer is this thing called software defined perimeter. Not many people have deployed this yet, but this is Google's BeyondCorp, if you're thinking about that. But it's basically insisting that someone go through an identity check and an authorization check before they get connected to any data in your network. So those are two things I would consider. Bobbie, you had another idea?

Bobbie Stempfley: So I think both of those are right, but I'm going to sort of circle back to nothing is free, and therefore you have to divide the problem into two parts. One is this is a supplier that you have a business relationship with. And for those go figure out who's paying who and then you have the opportunity to leverage that. And then those that you don't have a business relationship, that some user on your network has just gone and brought in to Joe, either whether it be an open source capability or something of that sort. For those, the first place I would start is any logical aggregation point. So if you're worried about open source software in your product lines, go to your Git and you know, figure out what's in there and figure out what products it goes along with.

Rick Howard: Excellent.

Joe Carrigan: I know that, that manifest you were talking about with a list of all the software, the open source projects in particular that are included in there, because nobody writes a lot of things in there. No IOT device provider writes their own TCP IP stack anymore.

Bobbie Stempfley: That's right.

Rick Howard: Bobbie, we got a question for you from Travis Fox. He says "Since you're on the Cybersecurity Canon committee, any books that you've read this past quarter that you'd recommend to anybody?" What have you been reading?

Bobbie Stempfley: Oh my gosh. So I haven't written my review yet, but I'm reading a book called Shadow War which is the study of Korea, Russia and China's cyber warfare activities that I really think is amazing. It was published by Columbia University Press and I just finished Ben Buchanan's book, also my review, which is actually sitting back there on signaling and statecraft. And both are really exceptional reads.

Rick Howard: Excellent offers. Joe, anything you're reading or listening to, doesn't have to be a book or anything crossed your mind lately?

Joe Carrigan: Christopher Hadnagy dropped a new version of the social engineering book. That's out.

Bobbie Stempfley: Ooh. Adding it to my list.

Joe Carrigan: I got to pick that up.

Rick Howard: That is a Canon Hall of Fame, right. His first version of that, right?

Joe Carrigan: Yes.

Rick Howard: I just finished reading Code Girls, OK, which is awesome, right. Most people are familiar with what we did during World War Two at Bletchley Park in breaking the enigma machine. This is the equivalent effort on the US side, breaking enigma on various codes, but mostly Japanese codes. The Japanese purple code. And the amazing thing from both efforts was that women were heavily involved on both sides doing the groundwork to break all those codes. So I highly, highly recommend it.

Bobbie Stempfley: And Rick if I can just, my grandmother was one of those women.

Rick Howard: Oh you're kidding me? That is so awesome.

Bobbie Stempfley: No. No. My grandmother was a WAVE, so they were navy auxiliary, part of it. Was one of those women sequestered at what is now the DHS headquarters, it used to be the navy cryptological facility, it was a Catholic girls school before that.

Rick Howard: Excellent. We got a question from Andreas Hoffitner,I'm sure I'm messing that up, right. But here's the question. "How do I continue to provide threat intelligence support to my organization after it has moved its infrastructure to the cloud?" He seems to be implying there Joe that you can't do threat intelligence because you've moved your outfit to some cloud. I don't think that's true, what do you think?

Joe Carrigan: Yes, I don't think that's correct either. You continue to get the information from the cloud provider and really threat intelligence feeds come from other parties that aggregate this stuff and provide it to you. They may charge you for it, there are other organizations like ISAACS, that if you're part of a certain industry then you can participate in the ISAAC to trade that information and get a hold of a threat intelligence to. No I don't think moving to the cloud precludes that.

Rick Howard: On CSO Perspectives, we just did,I don't know, six or nine episodes on cloud security stuff, and one thing we learned from that exercise was that the cloud provider provides lots of telemetry. So there's plenty of stuff to look at. Whether or not you can consume it or not that's a different thing, but...

Joe Carrigan: Yeah, now you're looking for the needle in the haystack or the grain of rice in the haystack.

Rick Howard: [LAUGHS]

Bobbie Stempfley: Needle in the stack of needles right?

Joe Carrigan: Right, needle in the stack. The right size needle that you're looking for. That's a better analogy Bobbie I think. [LAUGHS] In a haystack size of needles.

Bobbie Stempfley: Yeah. We're getting a phenomenal amount of data about the environment we have to use.

Joe Carrigan: Right.

Rick Howard: I would love to continue this conversation but we are at the end, OK. So ladies and gentlemen, thank you for attending. On behalf of my two colleagues here, Bobbie and Joe, thank you for participating, thanks for the excellent questions and we'll see you at the next Cyberwire quarterly analysts call, so thanks everybody.

Bobbie Stempfley: Thanks.