2022 Jailbreak Security Summit: Matt Rose: A Retrospective on the SolarWinds Supply Chain Attack.

Unless you have been living under a rock the past few years you have probably heard about the SolarWinds Supply Chain Attack. You have heard the jokes and seen the memes but do you know the whole story and the lessons learned from this significant attack? In this talk you will learn how complex the attack was and how hard it was to identify the root cause. Most people know about the attack but not the whole story. This retrospective talk will provide you with a detailed overview of the who, what, where, and how this attack occurred, was investigated, and remediated

Field CISO at Reversing Labs, Matt has over 20 years experience in software development, sales engineering, consulting, and technical leadership roles. During this time, Matt has helped some of the largest organizations in the world in a variety of industries, regions, and technical environments implement secure software development programs using cutting edge application security technologies. Matt's extensive background in application security, programming, and solution architecture has been key to many speaking engagements for organizations like OWASP, ISSA, and ISACA. In addition Matt has been the host of a successful application security podcast, been quoted in numerous industry articles, and is very well known for his video glass board sessions.

Transcript:

Matt Rose: Thank you very much. How's everybody doing? 

(CHEERING) 

Matt Rose: I have the honor after everyone's eaten a giant chicken sandwich or a meatball sub to present to you. So I will call you out if I see you sleeping. I will call you out because that was some awesome but definitely filling food. OK. My name is Matt Rose. I'm a field CISO at ReversingLabs. My background, I've been - and this is really hard to say - in the application security world for 20 years. I had a lot less gray in my beard when I started. My kids were babies. And now I have a kid in college and a kid in high school - pretty much had every type of role in application security, from consultant to SE to SE director and many other things, and just through the years just compiled a bunch of useless information in my head. And that's kind of what I say is I know a lot of useless information. And that's how I get by. 

Matt Rose: So please, be gentle. I try not to mess up too much. But I like just to be conversational rather than presentational. So the talk today is around SolarWinds and the supply chain attack against SolarWinds. Anybody heard of it? Show of hands. Yeah. Maybe? Also, there was a pandemic a couple of years ago, if you've heard of that. So we're going to kind of dive into supply chain - the supply chain attack associated with SolarWinds. I'm just going to give some food for thought. This is kind of like a recap, kind of an overview of the whole attack and just food for thought at the end. I'm going to surprise you with some additional content. I'm - you know, I was actually on the West Coast speaking at API World and talking about marrying API security with supply chain security as well. So I'm going to share a little of that content just to, you know, stand out from SolarWinds and really jump into maybe, you know, a new idea that you have in terms of risk of not only the applications but the application ecosystem with APIs. So with that, we'll get going. Really, the agenda today is the SolarWinds compromise timeline, SolarWinds supply chains malware families, you know, how the Orion software was actually compromised and how to defend against supply chain attacks. 

Matt Rose: And I'm going to kind of insert some interesting tidbits and some ideas that I have around this as well, so it's not just 47 slides that are coming up. No, joking - not that many slides. More conversation. And, you know, I really like conversation again. I'm going to, you know, ask you guys. There's no right or wrong answer. And you can call BS on me if you want. This is what it's about. It's about learning from not only the speakers but from your peers in the audience, too. So totally cool if you want to, you know, call BS or throw things at me because I'm really flexible and I'll dodge them. So... 

Unidentified Person #2: You're not. 

(LAUGHTER) 

Matt Rose: What was that? 

Unidentified Person #2: You're not. 

Matt Rose: I'm not? 

Unidentified Person #2: I'm dissenting. 

Matt Rose: Thank you, Captain Stubing. 

(LAUGHTER) 

Matt Rose: So really, let's talk about the timeline of the SolarWinds attack. And this is kind of interesting. And I got to take my glasses off because these are for distance. So just to kind of set the stage, attacks like this are multifaceted. One of the biggest things that I've seen in the industry is that everybody thinks there's a silver bullet, like I do A and B happens, that, you know, you do an action and you get a reaction. But it's usually like death by a thousand paper cuts. Two examples I have for you - well, the first analogy I like to say is, any attack, whether it's an application security attack at the source code or the running application, it's usually something that grows over time. The stupid analogy I like to say is, say I want to rob a bank. I'm a hacker. And I gained access through the front door to the lobby. Very interesting. 

Matt Rose: I'm in the bank. All I can steal there is some pens and some deposit slips - probably not what I really want. I want a bigger benefit to my illegal activities. So thinking about that, I do a little bit more research. Hey, and now I can get into the petty cash drawers behind the counter - a couple thousand dollars. Sounds good, much better than the pens and the paper. I really want to get into the vault. So getting into the vault is the game. So get in the lobby. Get in the petty cash drawers. Get into the vault. That's, like, an attack. And, you know, you'll see here by this timeline, if you're reading it, that it's a sequence of events. So you basically play the long game. You get into the lobby, but you hide that. You don't tell anybody about it. You get into the petty cash drawers. You don't tell anybody about it. You finally get those two steps and figure out how to get in the vault, now you have a formulated attack. And that's what most of these attacks are, is a step process. I got elevated privileges that gave me access to the network or the file system or the build system. And then I actually can do what I want. And that's pretty typical, too. Honestly, you know, I was actually hacked myself even being in cybersecurity for so long. Nothing to do with me, it was all to do with my bank. We won't mention the name of the bank. 

Matt Rose: But I have a savings account that was compromised and had no warning from the bank. And this is kind of funny at the end. No warning from the bank that some weird activity was happening in the savings account. No bills are paid out of it. Nothing is attached to it. Somehow, somebody got access to it, did a small deposit, a small deposit of cents and then took that money out - still didn't ring any bells with the bank. Then they took a ton of money out. Called the bank and their - I said, I was on - you know, I try and buy, you know, an iPad at 11 o'clock at night for my daughter for her birthday or something like that, I get an alert that that's scary. Someone taking out a ton of money out of the account that's never had anything taken out through an ACH transfer, that's something I should have been warned about. But the funny part about it was they immediately admitted, OK, we'll fix it. And I said, well, can you tell me what happened? No, we can't tell you that, sir. I'm like, I'm in the industry. I'd kind of like to know if there was something I did or something like that. Well, no, we have to protect the person that hacked you for repercussions against you. I mean, I was like, are you serious? So they're protecting - banks are now protecting the hackers, even though they may know who they are. 

Matt Rose: So with that, we'll talk about the timeline of SolarWinds and get into this. So starting around 2019, attackers start accessing SolarWinds. They're trying all the doors, all the windows, trying to get in, see what they can find. You know, they start injecting test code. Kind of like with my banking experience, they put a little money in, a little money out, a little money in, a little money in, and then took the money out. So, OK, I got something that works here. I can put money in. I can take money out. 

Matt Rose: So they did the exact same thing. They started attacking with test code. They stopped - once they figured out they could do that, they stopped it to cover their tracks. They're really not interested in - you know, again, they're not after the pens and the papers in the bank. They're really trying to figure out how to use this information for a much more targeted formal attack. So we fast-forward to about February of 2020. Basically, a hacking organization at a backdoor is compiled and deployed. So they found a way to get it into the system. We'll talk about exactly what they did in a second. 

Matt Rose: So and then in March, the estimated start time of distribution of the Solorigate backdoor, it was distributed through their process itself. So, again, it took them 2019 of September all the way through March. So they played the long game. They tried to get the information in there slowly and then hide themself. So then there was the distribution of the SUNBURST target profiling. And in May, the estimated start of attacks on - hands on the keyboard. One of the big things here you'll see in June is the attackers then said, OK, we've done our work here. It's now part of this built environment, and it's being distributed through the normal update process by SolarWinds. We're going to pull it out to kind of cover our tracks. And then those dotted lines basically say, you know, the continued hands on keyboard - they basically monitored and saw what was happening. 

Matt Rose: And then finally, in December 12th of 2020 - that's when it was actually found by - I believe it was FireEye and told to the world. And that's where everyone heard about the breach. Has any - have people seen this timeline before, or you know, talked about it? So let's see what happened here. So really, the SolarWinds build infrastructure was the thing that was compromised. A lot of times people think that scanning technologies at the app level, at the source code repository level is the place to go. If they're compromised the build environment, it's post build. It's post that repository type aspect. 

Matt Rose: So thinking about that, you know, the SUNSPOT was basically put into the MsBuild infrastructure. It was built as part of the product. And then SUNBURST was basically distributed out by Teardrop and Raindrop, depending on the system, to basically propagate the data. So again, supply chain - and I'm a big proponent of the supply chain should be part of application security. Supply chain is not something just for the SOC malware analysts, that application security and supply chain security should be one. And a lot of times people say, oh, I have software composition analysis. I look at all my packages via SCA. Well, the issue with that is if the build system is compromised, doesn't matter if you look at it prior to the build because it's something that was slipped in anyway. Make sense? 

Matt Rose: So really the process was SUNSPOT was injected into this MsBuild environment. You see there's a different tangent or a different stream associated with source control, the build process, which basically then compiled the Orion source code and included the new package or the new file that basically the hackers inserted. So thinking about this, it was a signed package. It was totally cool because it was part of the build. They didn't know this code was in there. And it's very hard because it was an unknown. They didn't know even what to look for. And they did a great job of covering their tracks. 

Matt Rose: So and from this standpoint, we have, you know, the SolarWinds Orion core business .dll and then the - basically the certificate information was all accurate. It was signed. It was accurate. Even if you looked at it didn't look like it was tampered with because it wasn't. It was basically - the code was part of that package and then it was signed by the normal processes. So all these versions on the bottom were basically pushed out. And then based on that timeline, they basically pulled out the code and then future versions were fine. They felt like they would get caught or got a little too greedy if they left it in the process - left it in that build process too long. So they're trying to not only attack organizations, government entities, so on and so forth, but they're trying to cover their tracks by removing what they're doing or kind of taking people off their scent, if you will. 

Matt Rose: So this little block of code, and it's really small for me, but really this is the backdoor code. So it was, like, twofold. The code ran to execute the backdoor malware. So it was twofold that there was code in there that basically, in inventorymanager.cs, that basically did the look up to see if everything was set up for the malware and if the malware would run. So first step was the code, and the second was the execution of the malware itself. So in Orion improvement business layer - sorry, really small - basically was run here and checked on this line that's the longest right here. 

Matt Rose: So instead of it just doing a direct lookup of select star from Win32 system drive, it basically hid that by basically trying to obfuscate and not, you know, draw attention to itself by using this opening of this - the zip file on the right side. So again, this is how they covered it and this is why it wasn't found immediately because virus protection and everything didn't see anything different because it was not something that was unique or different from what they normally see. So the forensics information - and this is kind of where my company came in - was really looking at, you know, things that changed. Why was something changed, you know? It's compiled on a certain date. It was signed on a certain date. But what happened in between, you know, from a SolarWinds Orion core business DLL to SolarWinds core, the different version, the hotfix. These things - there was file differences that changed that were very strange. But they were not alarming. 

Matt Rose: And this is where - it was not a DLL planted during packaging. It was not a stolen digital certificate. It was not a rogue release package. It was basically a valid package that had things in it that people did not know. So what happened once all this kind of hit the press? Well, from a developer side, they're basically saying to themselves, how was the back door implemented? How did this happen? You know, what was the sequence of events? They couldn't figure it out. And then, you know, which components are affected? You know, how was this propagated, what components, what aspects? And, I mean, Orion being a IT management software, it had access to the network. It had usernames. It had files. So it was a very, very easy target or a desired target just on the depth and breadth of the capabilities, in addition to the distribution that SolarWinds already had with their - I believe it was 16,000 customers. You know, which packages are they in? It's kind of like an Easter egg hunt at that point. Going through this application - you'll see how big this application was in a second. There's thousands of files, millions of files in some cases. So how do you detect similar attacks in the future? How do you understand what is happening so somebody doesn't do this again? 

Matt Rose: And that's where the supply chain analysis and reverse engineering of software either caught or homegrown really come into play because, again, when you say supply chain, most people really identify with open-source packages. But it's the open-source packages, it's the APIs, it's the homegrown code, the third party-developed code - all of that is part of the ecosystem. So on the buyer's side, you know, people are like, oh, crap. Do we have this, you know? Oh, we're a SolarWinds customer. Yes, we have Orion software. How do we actually identify if we're an issue, if we have an issue? Did we install the affected versions? I mean, it was, again, a little snapshot in time for some versions. Did they actually install those? Were those actually updated on their systems? Was it used? There was some organizations that said that they don't know what happened. But they felt like some emails and some things went missing. But they can't prove it. And then, you know, how to detect similar issues again? This is the ecosystem of supply chains. It's not just the here and now. It's about futureproofing. And a lot of this is around understanding the software not only internally developed, but your software. And I'm going to get into that in some cool videos in a little bit. 

Matt Rose: But, you know, the executive order that's talking about the cyber resiliency in the U.S. - and if you want to deal with the government, you have to self-attest that your software is free of this type of information. And they haven't gone as far to say that you have to use an SBOM. But they highly suggest it. And I think there's going to be language very soon that says, yes, this is what you need to do. And this is the format of the SBOM you need to provide. So getting the answers, this is how - kind of the process and looking at all the information, because it is like disassembling a million-piece jigsaw puzzle. You'll see that, you know, this is an example here of extracting a software package. So it's 458 files submitted for analysis. And this is how we go about the process. Total extracted files is 2.2 million. So trying to find a couple lines of code, that one little block of code, in 2.2 million files is very cumbersome. And the size itself is cumbersome as well. Antivirus just can't process a 34.5 gigabyte file. It's just too big. It basically falls over on itself. And that - you know, most enterprise software of SolarWinds' type is very robust in terms of its raw size. 

Matt Rose: And looking just at the difference of the files and doing a comparison, this is where, going through the exact list and looking at the information, it was identified that - I don't know what's happening with this. It was identified that this is the actual DLL that was actually compromised. So why developers couldn't detect SUNBURST - and this is kind of the lessons learned. Securing source control is not enough. A lot of people rely on their source code repository or GitHub or anything else for their source code repository to lock it down, don't give people access to things that's not there. And then scanning source code is not enough. How many people are familiar with SAST - static application security testing - or DAST, or software composition analysis or IAST, the scanners? I call them the star AST scanners in the space. They all look for different things. SAST is looking at the source code. SCA is looking at the open-source packages. IAST is looking at an application in a running state in the test environment. DAST can be in the test environment or in the production environment. And then one of the big ones now is API scanning based on the depth and breadth and the communication of APIs throughout any software ecosystem. 

Matt Rose: So - and scanning binaries for malware analysis is not enough either, you know? Backdoor code was hidden well. So if you don't know what to look for, you're not going to find it. In SolarWinds, that was the case. Nobody knew what to look for. So it doesn't matter how much you scan it. You can scan it every day of every week of every year, you would not have found this information without seeing some strange activity within an organization which caused the investigation. And, you know, backdoor code was novel. And what that means is never seen before. If you've never seen, you don't know to look for it. You don't know what it's going to do. So you have to digest it in a kind of open mind way to really understand what's happening and how to fix it and how to make sure it doesn't happen again. You know, why buyers couldn't detect it - they basically were asking - they weren't asking the right questions. You know, asking compliance questions is not enough. How many people - you know, if you're going to buy software or you're a seller of software, it's the dreaded questionnaire. You know, do you do this? Do you do this? Like, this isn't applicable to us. It's this subjective conversation. You're just trusting the person that's filling out the questionnaire. 

Matt Rose: So the right questions weren't answered. So compliance questions were not enough. You know, sometimes you don't know what's happening, and that's what was in the SUNBURST. So a lot of times I'll ask somebody, you know, being in OpSec, how big is that application? Do you know how many lines of code it is? No clue. They just know that it is mission critical, and it needs to be secure. But they can't tell you - I'm like, well, how many APIs are you using? Don't know. What open-source packages? Don't know. OK, so do you know anything about your application? 

Matt Rose: It's just a common thread that applications today are really living, breathing things. There's no such thing as a release. It's a Frankenstein's monster that - the knowledge of the application, the architecture of the application, the risks in the application are spread across multiple people. And that's actually what is propagating the risk in applications, is it's no longer just one person that understands the whole application. It's an ecosystem of people that all have to work together to understand the application, especially when you're bringing in cloud native development kind of concepts, when, you know, a branch is never actually brought together in a main trunk until it's deployed to that cloud infrastructure. So scanning source code for vulnerabilities is not enough. 

Matt Rose: Again, my background is primarily in SAS, in static application security testing. I work for - from the beginning from Fortify, the beginning from Checkmarx. I've seen all sorts of changes from waterfall development to DevOps to integrations to, you know, cloud native to microservices, all this type of stuff. You know what? In the 20 years I've been doing this, we're still looking for SQL injection and cross-site scripting after 20 years. Is that an education issue? Is it a lack of knowledge issue, or is it a code complexity issue? So just scanning the source code pre-compilation is not enough. 

Matt Rose: And then scanning binaries for malware is not enough. This is, you know, software composition analysis. You upload the open-source packages for analysis and review, and it says, oh, you got a licensing issue. Oh, there's a vulnerability in this. Oh, there's a new version that you need to update to based on some standard or spec or the inheritance of open source where open-source package inherits from another open source package inherits from an open source package. That's great if you're just concerned about the open-source package. But what about the rest of the ecosystem of the application - you know, the threat model of APIs and frameworks and libraries and everything else, not just the packages themselves? 

Matt Rose: And I know people argue all the time there. You know, I don't know if you've read the articles about open-source packages associated with software. And this is - I think it's kind of funny that, depending on the article, they say any modern piece of software is somewhere between 40 and 80% of source code. I'm like, that's a huge swing. Is my application 40% open source or is it 80% open source? And that's where that SBOM really comes into play because these industry analysts are guessing at these things based on research and talking to people. But software composition analysis and supply chain reverse engineering really allow you to understand truly what it is via SBOM communication. 

Matt Rose: So defense requires new analysis techniques, you know, decomposing the golden software image. Just because you built it, and it built successfully, doesn't mean it's free of malware, that it doesn't have issues. So you have to actually look at that golden software image. Describing underlying software behaviors - if something is behaving strange, it's worth investigating. And I know, last thing we want is more alerts, more warnings, more red flashing lights in our SOC or things like that. But there are ways to integrate this to make it an automated process that you can actually prove to your customers or to your internal stakeholders that the application is in fact secure and free of malware and specific risk. 

Matt Rose: You know, the behaviors are the big thing. If you see all of a sudden a privilege get escalated and a privilege get deescalated based on some logs, that's something to investigate. It's like, oh, I guess it's OK. It's kind of like the squeaking in your car. My brakes are squeaking. They're not squeaking anymore. I'm not going to take it in. In software, you really got to take it in. As soon as you see something change or something get a little bit obscure, you got to investigate because that's exactly what happened with SolarWinds. You know, in determining software deployment risks, you know, based on what cloud you're using, what protections are using the cloud? Are there credentials issues? So all these things working together really help with the overall supply chain risk. 

Matt Rose: You know, so you really need to look for known vulnerabilities, enable mitigations, certificate validation, sensitive information, third-party components, embedded malware, just like the bank and the hacking the bank. The protections have to be multi-level, too, because the attacks are multi-level. You can't - I always say, and I hate the term, please don't shoot the messenger, shift left in the software development lifecycle. There's no more left. Software development is done through an infinity loop. It's a DevOps process. Something is happening every time of every day during that process. There is no left. There is no right. There's just the process and the living, breathing piece of software that's constantly being updated. So thinking of it as just one entity or one thing you need to fix - you need to shift security and identification of risk everywhere within that process, from the developer environment all the way to that MsBuild or the build or CI/CD process. 

Matt Rose: So some example reports - this is kind of what you could actually look at in terms of the type of information once it's automated as part of the process where if you see these type of things based on a reputational database, you can see that there is a back door. We see the components, and this is the start and kind of the step to actually move forward with identification of the risk. So it's an example report. Again, this is unrelated to SUNBURST. This is just display information or presentation information. But you know, we have all these type of issues - certificates, security mitigation, sensitive data, third-party libraries detected vulnerabilities. And then you dig in to actually see these and correlate them and figure out which ones are most important to you. You know, you can actually - as you span this out, now you can actually see, you know, for example, detected Windows executable that does not implement safe exception handling vulnerability issues. So thinking about this, things are off. Something is off which causes investigation. And that's why using reputational databases that have seen things before or seen certain actions can lead you down the path of what they actually are 'cause they're not going to tell you directly. 

Matt Rose: So improving software quality - again, this is thinking about all the DLLs. And if you see a DLL, this one that's highlighted here in green, this libeay32, by looking at it and analyzing it from a reverse-engineering standpoint, this was identified as having risk in it and deeper investigation. It's not flashing red because, at this point, it's not known, but something is off with it. So having that investigation is very important. So - and then associated with SUNBURST - you can see this. Based on a reverse-engineering of the SolarWinds Orion package, these were the things, across the bottom, that basically identified the backdoor. So these things across the bottom, you know - contains reference to SHA1 algorithm .NET Framework classes, tampers with system shutdown. So a lot of this is around kind of covering the tracks and hiding what they were doing, why they were doing it and then getting out. 

Matt Rose: So a verified software release - again, defense in depth. When you're thinking about supply chain risk, yes, you should scan and identify risk prior to compilation based on the open-source packages NPM and PyPI. So you could actually - prior to the build process and then post-build process to make sure something hasn't happened in that step. Again, attacks are multifaceted. Your protections or your investigation has to be multifaceted. So prior to build and post-build, just to make sure that everything is really, really, really OK 'cause you may get caught if you're not actually looking at it in all aspects. That's why I'm not a big proponent of just shift left. If you think about this, source control, shift left would mean I'm just going to look for vulnerabilities in the source control. If the source is OK and nobody's accessed that, I should be good. No, it's constantly changing, and the threat landscape is growing exponentially, daily. You know, so if you're thinking about this, no known vulnerabilities, enablement of mitigations, properly signed packages, everything looks great, but what happens if you find something? Trustworthy behaviors and no embedded malware? Now you actually can fail to build or present something from being released based on what you find. 

Matt Rose: So now we're going to move into some bonus content. First, before we move into the bonus content - this is kind of changing gears a little bit - any questions? No questions. Crickets. Everyone's still digesting lunch, taking a nap, had too many beers already. No? OK, so we're going to shift up, and I'm going to make this a little more interesting for you. So one sec. I got to multitask with this thing. 

Matt Rose: So as I said, I presented at API World, and I'm known - I like to work with customers. I like to do whiteboarding sessions. I like to do interactive meetings to learn and discuss and erase. Well, the pandemic happened, and I got locked down, and I couldn't do that anymore. So I got creative, and I used this glass board to create content videos, post to social, company uses it. And I was actually tracked down at Black Hat this year - you're the glass board guy. Yeah, I do kind of stick out like a sore thumb with the red beard and the bald head and tattoos. But what we're really talking about here is I'm expanding out the conversation from supply chain risk to malware to the API ecosystem. So what is an application? Is an application just the deployable artifact? Or is it the interaction of all the pieces of functionality? You have the - you know, there's going to be an example here. You can see it on the board - ABC Bank, ABC Mortgage and Credit 'R Us. APIs are doing connections of, in this example, data and functionality. The trading of data is not only if the certificates and everything is authenticated correctly, but where does that data come from? The data that's being picked up by the APIs is, specifically, in the entity themselves. So we'll just play this real quick. 

Matt Rose: I hope everybody's enjoying the presentation by a guy that really looks like me, but he's over there, or is he over there? I don't know where the stage is set up. But let's talk about APIs and supply chain risk more from a glass-boarding whiteboarding aspect. So I got a simplistic example set up here with ABC Bank, ABC Mortgage, a subsidiary of ABC Bank, and another company called Credit 'R Us. Well, as we think about supply chain risk and APIs, really, applications are an ecosystem of things talking to each other. So let's hypothetically say that, hey, I go to ABC Bank, and I'm really looking for a mortgage. So the mortgage is taken, and then an API to basically start the process, to actually create the mortgage application, all the information associated with it, is passed off to ABC Mortgage. ABC Mortgage receives this information, starts your file with the data and the functionality passed off by ABC Bank. But they need to see if, hey, are you good enough to get this mortgage? Do you have enough money? Do you have enough credit? Well, the first thing we'll talk about is credit. 

Matt Rose: Now ABC mortgage calls Credit 'R Us to do a background check in terms of your credit. Hey, do you pay your bills? Do you have a lot of debt? So on and so forth. So that basically comes back with a reverse call or an additional call via API, from Credit 'R Us to the bank - to ABC Mortgage saying, hey, yep, you got good credit. Good on our end. But they said, well, you need some assets. We need to basically understand if you have the assets. How much is in your bank account? So ABC Mortgage then calls back to ABC Bank - and this is just, you know, high-level example of a flow. But I'll get to the point in a second - where an API calls back to do a look-up and check on the current balance of your accounts, how many accounts you have. Are they, say, what they are in terms of what you fill out? So we create this ecosystem of API calls from different functions in an organization, you know, ABC Bank in the subsidiary of ABC Mortgage or even a totally - a third-party company, like, Credit 'R Us, which is doing the credit check. And then, you know, hypothetically, you could even have more connections. And Credit 'R Us has much more - many more banks, and ABC Mortgage has many more, you know, banking branches or the ability to process through, you know, whatever means in terms of a mortgage. 

Matt Rose: All these APIs are creating this ecosystem, this interconnected network. But let's talk about the supply chain aspect of this. So do you know exactly what applications ABC Bank is using? Do they have an SBOM for commercial applications they're using? What about their home-grown code? Is it subject to malware? And the same thing happens for ABC Mortgage. Same thing happens for Credit 'R Us. All these things need to actually understand what you're actually connecting to, because if ABC Bank is using a third-party piece of software that has a potential of supply chain attack in it with malware inserted or you really don't know what open-source packages through an SBOM exist, this creates, potentially, an ecosystem of risk, where the data could be corrupted in ABC Bank that's sent to ABC Mortgage that's then sent Credit 'R Us. Or authentications are off, or things like that. 

Matt Rose: So when you think about APIs, don't just think about the functionality of the APIs - what it can do, what data it can access. You know, is it authenticated correctly? So on and so forth. But is it potentially talking to a compromised system through - in a supply chain attack? So this is going above and beyond just thinking about supply chain and API separately, how supply chain could eventually propagate itself through the interconnected nature of APIs. And I'll give it back to that guy or that guy or wherever he is sitting. I hope you enjoyed it. 

Matt Rose: So thinking about this ecosystem, who's responsible, do you think, in the end? Who's responsible for vetting the validity of their partner organizations? Anybody? So really, it's ABC Bank is responsible to make sure that anybody in their ecosystem of providers, partners, whatever you want to call them - that they're actually requesting that software bill and materials to work with them 'cause they don't want to basically just accept that they say it's secure. So it's really the responsibility of ABC Bank to vet every one of these and ask every one of them for an SBOM to verify and self-attest that there's issues. This is very similar to what the government is doing- is if you want to do work with the government, you have to self-attest and provide an SBOM to prove that your software is free of malware. So this is the first specific example. There's two other huge areas, in terms of risk, accessed by APIs. The first one that we're going to talk about - and these are directly coming from customers financial institutions I work for - is email risk around ransomware. 

Matt Rose: Welcome to the third video of the presentation. We've already had conversations around APIs and supply chain with data and functionality, and then we moved into file risks associated with APIs and supply chain risk. But the last one and one that's also very kind of important for people these days is email risk. So thinking about leveraging functionality in an application within an organization - ABC Bank, ABC Mortgage, Credit 'R Us. And again, we have our happy little end user here. And thinking about the APIs used to generate an email. And everyone's afraid of and cognizant of ransomware these days. Where does that email come from? Is that email spawned through good aspects of an application in terms of supply chain risk and proper configurations of APIs, so on and so forth? Or has something been compromised? 

Matt Rose: So you're thinking about an email being spawned by ABC Bank about the - your accounts or Credit 'R Us about your credit score or the status of your mortgage application, all going to an email address of the end user. They're going to trust that a lot more than just some blanket email that is being sent out by, you know, somebody they don't even know or somebody that they're trying to spoof, so on and so forth. But within those APIs, they're pulling data together across multiple systems, as this ecosystem we've been talking about, and create an email that may have a link or an attachment in there. If that link and attachment is compromised, people are going to trust, you know, something from an entity they know very well. So they have to be cognizant of the emails itself. So ABC Bank, ABC Mortgage, Credit 'R Us have to ensure the APIs they're using, their homegrown or third-party-developed applications, are secure in terms of the supply chain. How is this getting created? How is the data, how is the functionality being generated? Through APIs? Through email system? So on and so forth. 

Matt Rose: So email risk or physical emails are just as important as the vulnerabilities associated with files or with data or with functionality of an application being shared across the application ecosystem connected through APIs. So when you're thinking about supply chain risks and APIs, think about three main areas - it's that data. It's the functionality. It's the file. And it's the email. The email, again, is the most common pathway to a ransomware attack for an organization. Thank you very much for the third session. Enjoy the rest of the presentation. 

Matt Rose: Sorry I played them out of order. So we'll go to the data one now. We had to pull them out because, long story short, my laptop died. I was on a spare laptop, so T Mac and I Frankenstein-ed this all together to get it to run for you guys today. So I'll just go back to the last one. There you go. 

Matt Rose: Welcome to the next video in the presentation. I hoped you liked the first video, which was clearly talking about APIs... 

Matt Rose: That doesn't work. 

Matt Rose: ...Associated with functionality and - welcome to the next video in the presentation. I hope you... 

Matt Rose: I don't know what happened there. 

(CROSSTALK) 

Matt Rose: Yeah. 

(LAUGHTER) 

Matt Rose: How did that happen? Anyway, I'll paraphrase this because... 

(LAUGHTER) 

Matt Rose: ...Evidently, I did go to the bar before this, and I've held it together until this point. Well, so from a file risk, the file is a big thing, too. So you're thinking about, you know, a DocuSign type of environment or the mortgage application itself that comes over. Is there something embedded in there? And was there some process that embedded risk in that application, some sort of malware, some sort of backdoor in the file itself? And a lot of banks are really concerned about this because the way you communicate now through automated signatures and legal documents being, you know, signed with, you know, what is a representation of your signature - what if that information, a lot of that financial information, PII type data, you know, maybe trade secrets of contracts is sent to a third-party or a man-in-the-middle attack? 

Matt Rose: So really, the three areas when you think of supply chain in APIs is that processing of the ecosystem of data, of functionality, of emails and of physical files - PDF, DOC files, so on and so forth. So from the ecosystem of supply chain, it's not just about one package i the deployable artifact; it's about all the interactions that that package is capable of doing. And APIs are exploding these days. I mean, again, API World, there was so many different presentations about - with the risk and the proper implementation of APIs. The fact that there's a whole conference about API World and security right now is just, you know, proof is in the pudding that it's real. And the new vendors out there in the OPSEC space that are specifically around API security issues. 

Matt Rose: So with that, that's kind of my presentation today. I'm going to bring back to the slide deck and not drunk Matt. Drunk Matt's going to go away. The questions - and here's some background. You know, you can actually go to ReversingLabs. We have a lot of more information on white papers, data sheets, all about the SolarWinds attack, you know, food for thought in terms of implementing a secure supply chain, including videos and examples. And then you can always follow the content and the information 'cause we really are a research company, first and foremost. And our reputational database is probably one of the largest one in the world in terms of malware. And a lot of major organizations use us to really help identify SolarWinds-type issues. So with that, that's the wrap of my presentation. Any questions? Yeah. 

Unidentified Person #3: Nope (ph). 

Matt Rose: Way in the back, let me put my glasses on so I can see you 'cause it just looks like a blur back there. Yeah. 

Unidentified Person #4: (Inaudible). 

Matt Rose: Yep. 

Unidentified Person #4: (Inaudible) 

Matt Rose: So it really is - the question was, if people didn't hear, it's about the inheritance aspect. So, you know, package A takes functionality from package B, from package C, to package D, E, F - you only know that you're using A. You don't use - know you're using those other ones. But when you actually do download them, they're part of the package. So creating a - and the term I like to use is the world and the industry needs a dynamic bill of materials right now that is based on the artifact itself. Instead of looking at just the piece of code or that open-source package by itself, reverse-engineer and look at the entire deployable DLL or MSI or whatever it is for all the aspects of the application 'cause then that will have the secondary, tertiary - you know, how many levels deep it goes in terms of that inheritance. But that's a huge problem 'cause if there is a new zero-day out there that is called, I don't know, dog and cat and you look at your systems and go, I don't have dog and cat, but 12 open-source packages that you're using are actually inheriting - are actually using that environment, you have to actually know about it. So my biggest thing on that is - the top level? Look at it from not just the packages - the development packages but the entire deployable artifact. Is that - does that make sense or... 

Unidentified Person #4: What about for something that (inaudible) internet (inaudible). 

Matt Rose: Yeah. Well, that's a totally - that's where probably - if it's downloading that dynamically, it's probably using some sort of connection. And that would be found by a different technology in terms of runtime environment because, again, if you're reverse-engineering the artifact itself, you don't see runtime activity. So a lot of times, supply chain is - stops at that build environment, and then maybe you're using some sort of dynamic identification of risk for the running application, something that happens at runtime, or using - if it's levers to an API, use one of the API scanning vendors, like, I don't know, Traceable or Salt or Noname to actually see that type of callout. Does that make sense? 

Unidentified Person #4: Yes. 

Matt Rose: Someone else? 

Unidentified Person #5: Did you learn to write backwards on the glass board? 

Matt Rose: (Laughter) Everybody asks that. No, the - it does it for me. 

Unidentified Person #5: OK. 

Matt Rose: So the - there's a camera behind the glass where it looks like a 50-inch flat screen TV, and the camera image flips. So if you look at when I was writing, I'm not - I'm right-handed, not left-handed. So it's a good technology. It's actually called an E-glass. And the website's eglass.io. I don't get any royalties for that, but I think it's a great piece of technology to present. Any other questions? No. Well, thank you very much for your time. Hopefully you enjoyed it. Just one last question - do people like the glass board stuff? Is that interesting, new? Yeah. Cool. Thumbs up. OK. All right. Enjoy the rest of the conference. And thanks, everybody, for listening to me drone on today. 

(APPLAUSE)