Threat Vector 10.23.25
Ep 90 | 10.23.25

Shifting Security Left

Transcript

Krithivasan Mecheri: You have to meet where these tools are there, which is IDE. Like, you know, if you take it simply like a spell checker, right? Like, you should have a security checker as you are developing, or AI is helping code, recommend the, you know, corrections right there. That way, you are able to find and fix a lot of things ahead.

Sarit Tager: We went first, and the AI is changing the way application security is working, and you have to be thinking -- think about it differently. It's always better to plan ahead, rather than, you know, end in an incident in production. If you plan correctly, you will lead, and you will sleep better at night, by the way. [ Music ] 1

David Moulton: Today, I'm joined by two exceptional leaders from Palo Alto Networks. Sarit Tager, Vice President of Product Management, has led security strategy and R&D across multiple global security organizations, driving advances in application security and DevSecOps. Krithivasan, Senior Director of Product Security, is a veteran in AI cloud and application security, with deep expertise in software development life cycles and scaling security teams. Together, Sarit and Krithi bring complementary perspectives -- one from product innovation, the other from product security -- on how organizations can move from reactive fixes to prevention-first strategies; how AI is reshaping code security; and what it takes to evolve DevSecOps maturity. This conversation is about aligning speed with resilience, and why now is the time to rethink how we secure development. Sarit, Krithi, welcome to Threat Vector. I'm really excited to have both of you here today.

Krithivasan Mecheri: Thank you, David. Great to be here.

Sarit Tager: Hey, David. Great to be here.

David Moulton: And you're at headquarters today. Normally, you're coming in out of our Tel Aviv offices, right?

Sarit Tager: Correct. I'm in headquarters today.

David Moulton: Sarit, actually, let me start with you. What drew you into product management and security leadership, you know, really across so many innovative companies? We're fortunate to have you here with us at Palo Alto Networks.

Sarit Tager: Thank you. So I will start from my background. I started as a developer, and I was a team leader and kind of an engineer for a long time, handling all this crazy stuff of application security, just from the user perspective. So I spent a lot of nights trying to fix all these issues, usually before production. So it's kind of got into my mind that I have to provide something that will be easier for developers and will still keep the security level that we need within our larger organizations that we serve today. And once I understood that I wanted to go into product management, I kind of did different things around security for image scanning, for cloud security, for application security. I really feel that I'm on a mission to make it simple, and I won't say that developers and application security will live happily ever after, but at least they will like each other, and I'm very glad to have Krithi here as well. He's a great partner for our journey for a best application security product.

David Moulton: I think when we were talking about this episode, we were saying number one customer, right?

Sarit Tager: Yes.

David Moulton: The customer inside the house.

Sarit Tager: He is our number one.

David Moulton: Yeah.

Sarit Tager: He's -- he gives the best feedbacks and also opens most of the bugs.

David Moulton: Well, that's amazing. I remember giving -- or being given an opportunity to go into product management with IBM, and then, things changed and I stayed in design. But I, too, had that mission of, like, how do you find the balance between an incredible product but a significantly better UX? And that's probably better left to somebody with your talents than mine, so that's an awesome story. Krithi, number one customer, but let me take it over to you. You've built and scaled product security teams across cloud, AI, enterprise environments, obviously. What motivated you to focus on AI and product security here at Palo Alto Networks?

Krithivasan Mecheri: This is a fantastic question, David, glad you asked. These are exciting times for a security practitioner to be in, in the times of AI. Why? AI is a force multiplier. That means attackers are using AI. You see well-crafted phishing emails. Attackers are -- adversaries are using AI to, you know, find vulnerabilities in minutes. So it is very, very important for security teams to use AI to respond faster because, you know, you need to use AI to define AI-based attacks; that's one. The second, in Palo, we use AI for our products, our own products. We call it "drinking our own champagne," to, one, analyze terabytes of data, you know, detect anomalies, automate response. So my role ensures that we make our products secure using our own products. That's why I called it "drinking our own champagne," so that's one part. The second part is, hey, when you introduce AI into the development lifecycle, it opens up a different set of attack vectors, right? So data poisoning, prompt injection, model inversion attacks, so how do we inject AI early into the development cycle, AI security early into the development cycle; give the engineering teams the necessary security tools and processes to achieve the security by design within the new development process? So that is why I'm saying it is super-exciting times to be in here right now, to be in this industry when AI is in the forefront.

David Moulton: Absolutely. I mean, it's a big, big job and an important one, but I think it's also one of those things that lights us all up here. And, man, I hope that we're able to get it right because it's moving so fast, and there's so many of us counting on the work that both of you are doing, and I think that's why I was so excited to have this conversation today. So today, we're going to be talking about prevention first security, AI's effect on code velocity, and what it takes for organizations to drive cultural and strategic transformation in DevSecOps. Both of you have built these really impressive careers shaping product security and DevSecOps. Sarit, let me start with you. What's a key lesson from your journey that shapes how you approach application security today?

Sarit Tager: So it's kind of a changing method, I would say, in a way, because if we think about application security back in the days, then developers were writing a code, and then, the code was scanned after the fact. Usually when there were builds, the builds were not that often, and then, you will get your problems only later in the process, if at all. And in most cases, and this is also coming today, things were not blocked within the PRs or the build process because when you think about developers, they are very unique. And usually, you don't want to hurt their velocity. And you say, okay, we will develop as fast as possible and see how we can actually mitigate some of the problems later in the process. However, if you think about it, if problems arrive to production and you try to fix them after they are in production, you usually meet a developer like three months after he actually wrote the code, and he may not remember what he wanted to write there and what was the meaning of the code or how we should fix it. And eventually, you spend much more time on trying to figure out who has the ownership; how to fix it; build it again; ship it again to production. And basically, this was kind of a vicious cycle till now, that everything is going to production and you have to figure out what is the problem, how to fix it, who is the owner? We do see a lot of customers understanding that it's not about trying to fix it in production but trying to prevent it beforehand, because if they meet the developers at their system, whether it's in their IDE or when they submit a pull request, then the developer itself can say, oh, okay. I know how to fix it. I can do it now. I don't need to wait for something to be shipped and deployed and only then fix it.

David Moulton: Yeah.

Sarit Tager: And this even becomes something which is much more urgent, in a way, if you think about all the vibe coding and the fact that the developer is no longer the only responsible for the code that is being generated. So you must make sure that you fix it as early as possible or even create it securely early as possible.

David Moulton: You know, when you're talking about this, I wish that that mentality had been available back in seventh-grade math for me. And it occasionally showed up when Mr. Gawker [phonetic], shout out to Schmucker Middle School [phonetic] there, would sit down with me and watch how I was going through math problems. And he'd correct me at the point that I made the mistake, not at the point that I turned in the work or the homework, so that I understood how to stop making that mistake in real time. And what you're talking about has a different level of implication. When you're shipping code, when you're shipping an application and you're counting on your QA, you're counting on the teams after the fact to catch it and route it back and get it fixed, not only are you losing velocity, but it seems like you're enforcing the same mistakes to be made over and over with a really slow feedback cycle. So it's amazing to think that that mentality shift is available to the teams here, and maybe it'll show up in middle school math classes everywhere. Krithi, let me take it over to you. How does that resonate with your own journey building and scaling product security teams?

Krithivasan Mecheri: I agree with Sarit 100%. I used to be a developer before I moved into security, so I understand the landscape. Traditionally, security, you know, you will build the application. You go to go live, then security comes and scans and says, hey, you know, here's the 50 things you need to fix. It became kind of a blocker for things to move faster. The biggest challenge was always integrating security early in the lifecycle to address the vulnerabilities. So our shift-level approach, with great partners like Sarit building amazing products, is to streamline AppSec; integrate it early into your development process, IDE, you know, pre-receive commits. You know, we're integrating with source code management systems. Give real-time visibility to the developers who are actually the owners of their applications, and as Sarit said, how do we automate? Instead of saying, hey, here's 10 things, can we automate fixes? Can we enable developers, you know, to fix while they are being -- developing the application? This kind of was a success here. And the second thing, what we learned is the toughest thing for any security team is to find the right owner. When you integrate early, you are able to address this with the right stakeholders and the right partners, which actually helps you accelerate the vulnerability-fixing process too. And this is kind of the central mission of my team at Palo, make sure all the vulnerabilities are identified, fixed, and addressed before even it gets released. So that's kind of, I feel, is the right way to implement shift-left in the industry.

David Moulton: So let's jump into application security posture management, or ASPM for those who have a lot less time, and then, thinking about some security backlogs. A lot of organizations face this overwhelming security backlog, millions of alerts, with maybe only a fraction of those remediated. Why does that problem still exist and persist so prevalently?

Sarit Tager: So maybe I will take this one. Usually, when you write code, you write more vulnerabilities. Like, code comes with vulnerabilities that come with more security problems. When customers or companies look at this and say, okay, I have too many and I cannot actually block anything, this is quite a challenge because they say if I will block something, it will block the entire pipeline. So I will block the entire velocity of the developers. I will never be able to shift anything to production, and this is super scary if you think about it. When we talk about how this should be managed, it's a journey, and application security maturity is a journey. It's not something that you do once, and then, it just solves everything. And the idea is to build it in steps. So, for example, the first step should be -- and this is something that was implemented also in our company -- is let's first stop all the new stuff from coming in, and then, try to see how, gradually, we can also fix some of the backlog and some of the technical depth. But try to fix first all the new stuff is very important as a method to be able to actually solve this problem. So this is one issue. The second one, I would say, is more about remediation. Like, how do I do remediation? Is it simple? Do we give the users all the steps to remediate things? Do they know how to remediate it? I would say that when you look at kind of the evolution of all these application security scanners, they focused a lot on finding the problem, but not necessarily helping remediate it. And I think these two barriers of just overwhelming number of issues, and the fact that the remediation is not simple is something that's kind of become very hard to maintain and fix and keep the level of security needed for a company.

David Moulton: That makes sense. Krithi, let me take it over to you. How do you help executives and security leaders reframe that backlog reduction as a real strategic priority, rather than just this, like, technical work that needs to be done and maybe ignored?

Krithivasan Mecheri: To actually help frame a security vulnerability to a business executive, we need to talk to their language. We need to translate vulnerabilities into a business risk or a reputation list or a loss of customer trust, so I'll give you an example. Say, instead of saying, hey, there are 50 vulnerabilities in your product or an application, you should say, hey, here are 5 or 10 top issues. If it doesn't get addressed, it gets turned into a product vulnerability, which in turn results in loss of customer trust or a competitive disadvantage, okay? So that kind of helps, saying, hey, this needs to be prioritized along with the release. That's one way to look at it. Second, is every vulnerability a priority? Probably not. One way to do that is to have smart prioritization. Is it external facing? Does it have customer data involved? Is it, like, a public-facing application? So you add this framework to actually say, hey, out of this 500, here are the five things you need to go address before you go live. See, that kind of helps get you executive buy in. Second, for the engineering or engineering leaders, you need to frame this differently. Instead of saying, "Hey, I need you to fix these 150 Jira tickets," you're going to say, "Hey, let me give you, you know, golden templates. Let me give you secure base images. Let me give you automated PR, so that you can actually burn down the backlogs. You can prioritize the, you know, business-critical bugs to go fix, and then, you can move faster without compromising security," right? So in Palo Alto Networks, we actually embraced this shift-left strategy, integrating our own products into the CI/CD pipeline, which helps developers get real-time vulnerability visibility; enables quick fixes, drastically reducing the security backlog; and then, boosting developer productivity, too, which is kind of always, you know, not much spoken about, which I think is a very important metric where security teams can use it to get more adoption. [ Music ]

David Moulton: I'm wondering how you balance that prevention-first mindset in an environment that is so intensely focused on the build and deploy of new AI capabilities. It sounds great, but there is a lot of pressure from the business to not just keep up, but to land that innovation. What's your perspective on balancing the speed with prevention that you're talking about?

Krithivasan Mecheri: So I would frame it a little differently. So the key is you have to go from a manual and reactive process to an automated and proactive process. What do I mean by this is, for example, a developer normally develops codes. He commits into a source code management system, a build gets run, and then it gets deployed. The more closer you get to the developer, the better outcome you're going to get. That means you have to, one, integrate the scanning well ahead before something happens. That means it has to be part of your commit process or a build process. Second, you have to empower developers with the right frameworks, right? As I said, golden templates, security, you know, tools, process, integrated early gives you that advantage. Also, automating PR fixes, like what our tools do, gives a lot more developers power to address this, and then, this begets more a reactive process to a more proactive process. And then, more from manual, where security teams used to do it at the end of the cycle, to a more proactive, where it's constantly being scanned and constantly being addressed before it gets even built and deployed. So effective in TMBR is automate, automate, automate as part of the SDLC.

David Moulton: Sarit, can you talk to me about how ASPM and context-aware controls help ease that transition?

Sarit Tager: So the idea is that when you think about the overwhelming amount of issues we have in application security, you think about the code. And usually, when code is being written, there are some things that go into testing. There are some things that don't necessarily go into your application, and if you look only at the code, it will be hard to understand what needs to be prioritized. There are, of course, things that can be done on the code side. When we talk about ASPM, the idea is that we take the prioritization from everything, all the signals we have within the system, and the fact that we also see the entire production environment; that we know exactly where the container or the VM is residing; whether it's a production environment or a development environment; whether it has an access to the internet or not; whether it contains a sensitive data, yes or not; if it has an excessive permission; so there are a lot of signals that we can take and say, this is the environment in which your code is going to be deployed. And because we know that, we can say, okay, so issues or problems on this specific code base has to be solved earlier than others. And this is a major part of trying to prioritize the way we look at things, and it's also the way we do the policies. Like we take all this information into the guardrails and say, let's look first for all the things that are more risky than others. And ASPM comes here and says, not everything can be solved in the first hand. We have to prioritize, in a way, and the way to prioritize is to actually understand how your application is going to look in production and take all this information back to the developer, back to the PR process, and understand exactly what needs to be done to be able to solve the important things first.

David Moulton: All right. Let's shift our gears and start to talk about AI and code security specifically. You know, I know that AI is rewriting the rules of development. It's accelerating code delivery by at least 10x. Sarit, what security risks do you see this creating?

Sarit Tager: So this is a great question. Actually, it's not one, but it's a few problems. So one will be that I would say the shift in responsibility. If you think about it before, the developer was the one responsible for the vulnerabilities or the problems that were getting into the code. Now we have some kind of an agent that is writing the code, and the responsibility is kind of being shifted between the developer and the agent. Sometimes the developer accepts or rejects -- approves or rejects things from the agent, but this will be the first one. It's not just a shift in responsibility; it's also a shift in knowledge. If I -- like, I was a developer a few years ago, then I knew my code. I understand exactly what I did, and hence, I could have remediated it. Now that an agent did it, this is kind of a surprising one. The second one will be if you think about agents, and the way they -- even if they kind of have, like, MCPs and you add your scanners and you say, yes, let's do a scanning, it has to be done as a prerequisite and not a post one, a post scanning like we do today. Today, we first write the code and then do the scanning. In AI coding, vibe coding, it has to change. It has to be part of the way you generate the code. And then, come some questions. For example, will the agent adhere to the requirement by the scanners? It's not necessarily happening, so this is something that we need to kind of take care of. And another one is the entire new attack surfaces that are being created by this vibe coding. Think about how [inaudible 00:23:59] are working, just an example, of course. They have an agent. They have the LLMs that they are working on. They have their MCPs. Not all of them are secure. You may find yourself -- and Krithi can say that -- with a lot of MCPs, servers that are not approved by your application security practitioners and they are not part of your organization approval, the whitelist, or something like that. And then, you may find yourself in a way that things -- the agent will do things in the computer that you are not protected from. So first will be the problems of the post-scanning, the fact it's no longer only the developer that is responsible for. The second will be the new attack surface. Like, there are MCPs, new supply chain attacks, so there are a lot of different areas in which vibe coding is bringing value by generating code velocity -- very important. But it also has a risk in which you have to protect your environment in a better way and make sure that you know exactly what is going to be applied to your code; how do you protect the users, so a lot of interesting stuff ahead.

David Moulton: Yeah, you know, I was just on an interview with a research firm asking about AI and its usage inside of marketing organizations. That was specifically what she was looking at, and I think that alongside our, like, different corporate trainings for security policy or for, you know, how to be inclusive, how to make sure that we're upholding our ethics and our values as an organization, there's got to be AI training. And I think what you're talking about is beyond just the regular security training of don't click on the phishing link and, you know, stay away from some of those things that we see as patterns. But what are the types of things that are going to happen when you are using AI tools to accelerate your ability to succeed in a corporate environment, and what are the types of things that you open the business up to? And maybe that's not necessarily the path that both of you are, you know, thinking about, but I think if we don't see that in the training within the next six months, that there's been a massive miss because the amount of risk and the ease at which the risk is taken is -- it's just -- it's everywhere right now. It's actually quite wild to me. Krithi, I want to switch it over to you. From your vantage point in AI security, what vulnerabilities do your teams see that are often overlooked in that AI generated code?

Krithivasan Mecheri: That's my favorite question, actually, out of -- in the podcast. So here's my take, okay? So with the amount of choices developers have of using different models, different code generation tools, right? These are all built on functional correctness, lack, sometimes, security context, so leading to many code related vulnerabilities. Most of the times, we have seen input validation missing; you know, weak access control; we have a hard-coded credential. So it becomes super important, one, to understand how these tools work fundamentally. Second, also, we have seen, because of hallucinations, models and tools recommending packages which don't even exist, which leads to typosquatting, recommending packages, older versions, or vulnerable packages. So it kind of opens up an interesting perspective because you have to now detect everything at scale, and that is what I see as a challenge and an exciting problem to go solve from a practitioner's point of view.

David Moulton: Let's talk about economics and security strategy for a minute. Fixing issues in production, I think I've seen a stat that says it costs 100x more than just preventing them earlier. Why do you think organizations are still so accepting of that kind of inefficiency?

Sarit Tager: I would say mostly because they are afraid to block the velocity of the business and not create -- it's kind of blocking the things you know rather than the things you don't know. If you have vulnerabilities within production and nothing is being exploited, then you feel safe in a way. But if you think about it, today, developers spend a lot of time on security problems that come from production. And again, they are at the end of the chain. Like, trying to get to the right developer is really a challenge. It's a challenge by the app tech teams. It's not a challenge by the developers. Like, the app tech team needs to understand, okay, this incident, or this exposure problem, what happened? It's actually something that we need to track back, understand who is the owner, or see if we can fix it. They're probably already working on the next version. It's not the one that is currently in production. So I think because it's not in one place, then you don't necessarily see the impact in one group. But if you do look at it as a holistic way of things, you will see that, while you may spare your developers by fixing things when they were part of the development cycle, you will spend their time while they do it on the production. It's the same as bugs, if you think about it. Like, the reason we have QA and testing is because we want to make sure that we find things before they go into production and before it's actually been malfunctioned on production. This is the same for vulnerabilities. We need to shift it left, and I do believe that once all these tools will be much more integrated, remediation will be simple, and it won't take that much time; then the organization will move to working much more in the developer area rather than in production.

David Moulton: Speed versus security is often painted as this zero-sum game. Krithi, what strategies have you seen work to achieve both speed and security?

Krithivasan Mecheri: So the first thing is to make sure security is an enabler. So, for example, how I talk to executives is to say, hey, by integrating security early and addressing vulnerabilities ahead of time, you save x numbers of developers' time to go actually develop new features, which will lead you to faster to market, which will lead you to competitive advantage, right? The framing of how you implement shift-left to executives is very important. Second, security is a differentiator. For example, if you do security well, if you do security well ahead of time, a lot of these security standards, if you build it right, is actually a very, very essential differentiator for business. For example, getting FedRAMP certification, getting IRAP certification. So a lot of these all have overlaps, so building as this part of design and finding and patching everything ahead of time is definitely a business enabler. Third, empowering developers, which is very, very important. As I said, doing shift-left is not only security. It's actually giving the power back to the developers to actually address these security vulnerabilities ahead of time. So combining these strategies, I think, is the right way to go.

David Moulton: Krithi, you've built security champions inside of dev teams to drive cultural change from within. Can you give an example of one thing that you found works really well?

Krithivasan Mecheri: Identify the right people. There are always great people with passion for security in engineering who has a lot of influence. Identify them reward them, give them visibility; empower them with tools, and then, you know, be -- train the trainer. Make them your wise eyes and ears. That's the best way to scale, and I've seen it work fantastically in Palo Alto Networks. [ Music ]

David Moulton: Thank you both so much for this really fascinating conversation and a little bit of stress as I think about the challenges that you're both trying to solve, and I appreciate that you took some time out of your day to come on Threat Vector today and talk to me, you know, on this podcast.

Sarit Tager: Thank you very much.

Krithivasan Mecheri: Thank you, David. It was a pleasure.

David Moulton: That's it for today. If you like what you heard, please subscribe wherever you listen, and leave us a review out on Apple Podcasts or Spotify. Those reviews and your feedback really do help us understand what you want to hear about. Reach out to me at Threat Vector at paloaltonetworks.com. I want to thank our executive producer, Michael Heller, our content and production teams, which include Kenne Miller, Joe Bettencourt, and Virginia Tran; original music and mix by Elliot Peltzman. We'll be back next week. Until then, stay secure, stay vigilant, goodbye for now. [ Music ]