A subtle flaw, a massive blast radius.

Dave Bittner: Hello, everyone, and welcome to the CyberWire's "Research Saturday." I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down the threats and vulnerabilities, solving some of the hard problems, and protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us.

Yuval Avrahami: What actually brought it to our attention was an actual threat actor that managed to take over an AWS GitHub repository using another CodeBuild issue. We saw this and we thought it's, you know, pretty insane that it's possible to do something like that, and that's what originally led us to look in this direction.

Dave Bittner: That's Yuval Avrahami, Vulnerability Researcher at Wiz. The research we're discussing today is titled "CodeBreach: Infiltrating the AWS Console Supply Chain and Hijacking AWS GitHub Repositories via CodeBuild." Well, before we dig into CodeBreach itself, could you explain to us how is AWS CodeBuild normally used and what makes it an attractive target?

Yuval Avrahami: CodeBuild is a CI/CD service. So it's normally used to trigger builds on source code. So what you would normally do if you have a source repository, a code repository in GitHub, for example, you would connect CodeBuild to that. So whenever you do something like a push or a pull request or you have a new feature and you want your automated test to run, those could run in CodeBuild. And what makes it an interesting target is that for CodeBuild to interact with GitHub, with the source repository, for example, to tell it if your build failed or something like that, it actually needs some GitHub credentials. And often those are very powerful. And that means that if you are able to compromise a build that's running on a CodeBuild, you could end up hijacking the entire repository. So that's why it's an interesting, you know, target for us and for attackers. We saw attackers actually go after it as well.

Dave Bittner: Well, take us through CodeBreach. What exactly is going on here?

Yuval Avrahami: Because the consequence of having a build being triggered for your pull request is that you now have code running in CodeBuild next to the GitHub credentials. So this is, you know, something you wouldn't want anyone to be able to have. CodeBuild has this concept of webhook filters, which basically mean, in simplicity, it's just a list of GitHub users that are allowed to trigger builds, right? So that actually is a valid way to do it. Like it's a bit annoying from, you know, just a user experience perspective to have to like fill in this list of every time someone joins your team, you search GitHub user ID, you need to add it to the list. When we saw this, we were like, our organizations are actually implementing this. Like it sounds like a bit of a hassle, you know? So this is how we started to look into this. We thought we're going to look about, we're going to search for CodeBuild projects. And we -- our hypothesis was that people are probably just lazy, you know, they probably are not implementing this. So we went on to search for like CodeBuild projects that were configured as public. So you have this setting which makes a CodeBuild project public, meaning anyone can see its settings. And that allowed us to actually search for, you know, CodeBuild projects and see their configurations and see if, you know, people are actually setting this list or are they being lazy. And we found a couple of AWS GitHub repository, like a pretty major one, and we'll talk about one of them later on, which were connected to CodeBuild, and they actually had this list of users. So we were like, okay, good for AWS. You know, they weren't being lazy. They actually narrowed down who could trigger builds in their GitHub repositories. So at first, it looked like a dead end. But when we looked at this, you know, list again, something stood out. The separator between the numbers in the list, it wasn't like a comma or a space, it was a pipe character. Which is, you know, quite odd for a normal list. And that was actually the clue that this is actually not a normal list, this is a regex expression. Which basically meant that it will not only authorize the user IDs in the list, but because of how regex work, it will authorize any user ID that contains an approved user ID.

Dave Bittner: [Laughing] Okay. Wow. So how does this differ from a more traditional cloud misconfiguration?

Yuval Avrahami: So I think, first of all, it's not that clear that it's a misconfiguration. Like the people who configure it in AWS assumed it was a queue. And also us, when we saw this list of numbers at the beginning, we weren't sure like, okay, this is like -- they specified which accounts can actually trigger a build. So it looks like it's very subtle. I'd say it's what makes this interesting. It's why it went unnoticed for a lot of time. And the impact here is really crazy. Because of this subtle misconfiguration, you are able to actually take over key AWS GitHub repositories. And we're talking like some of the most crucial GitHub repositories that are used all across AWS environments.

Dave Bittner: We'll be right back. Well, can you walk us through an attack scenario here from the attacker's perspective? How do they go about doing what they want to do?

Yuval Avrahami: So the way to go about it from, like, if you're an attacker that wants to actually exploit this, you need to open a GitHub account. which has an ID that contains one of the approved maintainer IDs. And this is actually not that simple. An ID like that is available for registration, something like once every five days. Because you need the ID to be like to have an exact match of an approved maintainer ID inside it. So every five days, something occurs that we internally called it an "eclipse." You know, when an ID is exactly a super string of an approved ID, and when that moments come out, you just need to flood GitHub with a lot of user creation requests, right? Because users in GitHub are created all the time. If you don't create a lot of users at the precise moment where the ID is out for registration, like someone else is going to take the, you know, this special ID. And so we figured out a couple of ways to actually bypass. It's not really bypassing GitHub rate limits, it's just a couple of avenues which GitHub doesn't rate limit at the time. And we find a way actually to create a lot of GitHub users at the precise moment to actually capture the attacker user ID. And once you have this user ID, the attack scenario is really simple. You just open a pull request to the GitHub repository you want to compromise. Once you open it, because CodeBuild sees your GitHub user ID, and because of how the filter is implemented, a build will be triggered from your pull request. And now you have code execution in a build which has GitHub credentials. So you have those credentials, and at that time, you can just do whatever you want to. You basically are an admin of the repository. You can push code, open pull requests, exfiltrate secrets. And we actually saw someone do something like that about a month before our research. So we know that people actually, you know, are using this mechanism. It was a different CodeBuild issue, but, you know, the overall attack pattern is the same. You open a PR, you abuse some misconfiguration to trigger a build in CodeBuild. And once your build is running, you can take the GitHub credentials, right? And just, you have complete control over the GitHub repository, you can release your own code, you could, you know, push malicious code. The main thing is just doing this without being noticed, right?

Dave Bittner: Are there any particular kinds of organizations that would be most at risk from this issue?

Yuval Avrahami: First of all, anyone that uses a CI/CD service can have this issue. The main thing is whether your repositories are public or private, right? Because if you have private repositories only, for example, the only one who sees the repositories and can attack them are the internal users, right, your own company users. So the risk is much lower than if your code repository is public. And that exactly is what happened to AWS. They, you know, open source their code. And unfortunately, the side effect of doing that is that anyone can see, you know, your code and sometimes the CI/CD configurations. So if you are an organization that, you know, runs CI/CD builds, on public repositories, you should really take notes about it, you know, regarding issues like this. Because what makes them crazy is that the attacker has no prior access to your organization at all, right? And he just submits one pull request, he gets his build running, and suddenly he's an admin of one of your key components, right? It's such a supply chain nightmare. It's a really, really interesting risk.

Dave Bittner: Now, how did AWS respond once you disclosed this issue?

Yuval Avrahami: Oh, they were great. They were really fast. Within, I think, 48 hours, they already mitigated all of the, you know, problematic filters. And they actually released a new feature, which automatically, instead of having like this list of user IDs that you need to maintain and keep track of, AWS now automatically has a feature that checks what is the actual, you know, permissions of the person who opened the pull request over the GitHub repositories, right? So it automatically detects if you're an owner of the GitHub repository, and only if so, it will allow a build to trigger. And this is such a good improvement. But the thing is that this is the default for new CodeBuild projects, right? So if you have CodeBuild projects from before this issue, you need to change your settings. And this is the main like action item from this issue. Go to your CodeBuild settings and enable this feature. It's called "pull request comment approval." And it really is like an easy click, and you're much more secure.

Dave Bittner: So the fix is available, but you have to know to do it and then actually go ahead and do it?

Yuval Avrahami: Yeah, yeah. Like AWS fixed the repositories that they have that we found are vulnerable, right? But it's problematic for them to just go inside, you know, existing projects and change their settings without the customers knowing about this, right? So they automatically enable this for new projects. But if you have an existing project, you should really look into whether this setting is enabled.

Dave Bittner: From a high level, are there lessons that security teams can take away from this in terms of trusting managed cloud services?

Yuval Avrahami: Yeah, it's a tough one, right? Because you don't know that if you like implement this yourself, you're going to do necessarily a much better job, right? But I think this falls into the configuration of the cloud service. And in these areas, when you have like a lot of recent attacks -- and the industry is a bit behind, like CI/CD security -- I think you really need to like double check your configuration. And if there are any way for you to reduce the privileges, you just need to do it, right? CI/CD is such a problematic vector right now, we see countless attacks. So the main point is you want to close the biggest, you know, attack surface right now, which is public repositories. So you really need to look at the CI/CD services of your public repositories and try to secure them first, because they are open to the world. And this is what makes them good targets for attackers. [ Music ]

Dave Bittner: Our thanks to Yuval Avrahami from Wiz for joining us. The research is titled "CodeBreach: Infiltrating the AWS Console Supply Chain and Hijacking AWS GitHub Repositories via CodeBuild." We'll have a link in the Show Notes. And that's "Research Saturday," brought to you by N2K CyberWire. We'd love to know what you think of this podcast. Your feedback ensures we deliver the insights that keep you a step ahead in the rapidly changing world of cybersecurity. If you like our show, please share a rating and review in your favorite podcast app. Please also fill out the survey in the show notes or send an email to cyberwire@n2k.com. This episode was produced by Liz Stokes. We're mixed by Elliott Peltzman and Tre Hester. Our executive producer is Jennifer Eiben. Peter Kilpe is our publisher, and I'm Dave Bittner. Thanks for listening. We'll see you back here next time. [ Music ]