Decoding XDR: Allie Mellen on What’s Next

Allie Mellen: There is nothing more important than understanding what your point of view on whatever situation you're a part of is, and being able to articulate that in a way that makes sense to others. That's what the values conversation is ultimately about. That's what I expect and hope for from vendors whenever we do a wave evaluation. That's what I expect and hope for from customers whenever they're talking about what they want a vendor to do differently. And so I hope that everyone can take away from this conversation that if you are able to think about and develop your unique point of view and back that up with actual data and understanding of how you're going to get to the outcome that your perspective is giving you then that will lead you in the right direction. And I've seen that at least in my life for my entire life. [ Music ]

David Moulton: Welcome to "Threat Vector," the Palo Alto Network's podcast where we discuss pressing cybersecurity threats and resilience and uncover insights in to the latest industry trends. I'm your host David Moulton, director of thought leadership for Unit 42. Today I'm excited to be joined by Allie Mellen, principal analyst at Forrester and a thought leader in the field of security operations. Allie specializes in XDR, detection engineering, and the evolving security technology landscape. Her research and insights have helped countless organizations navigate cybersecurity threats more effectively and she also shares her analysis with the broader community through her popular newsletter "The Latest Breach." Our topic today is decoding XDR. As XDR rapidly evolves, it's becoming a key tool for security teams to consolidate data and better detect and respond to cyber threats. But what's next for XDR? And how can organizations separate hype from reality? Stay with me today to hear from Allie how she answers this challenging problem. Allie Mellen, welcome to "Threat Vector." I'm really excited to have you here on the show.

Allie Mellen: Thanks so much for having me. I'm thrilled to be here.

David Moulton: I want to start by asking you a quick question about your newsletter "The Latest Breach." What inspired you to start that and what do you think the biggest value is for your readers?

Allie Mellen: Oh. So I think that there's so much going on all of the time in cybersecurity that can be difficult to kind of look back and dig in to some of the things that have happened in the space and why they're important. And so what "The Latest Breach" is really looking to do is let's take a look at some of the breaches and some of the cyber activities that have happened in the past several years and first off give a really easy to understand explanation of what happened and why because I think that's one of the biggest gaps is there's just so much confusion and so much difficulty for people of all levels to understand what's happening from a cybersecurity perspective. And then also let's use it to help make the case for why cybersecurity is important and to help communicate that to other people in either your organization or just in your lives in general. I know that I get a lot of questions from family members and friends that are like, "Hey, what happened here? Why did this cyber attack happen?" Or, "What does it mean for me?" And the goal is especially with "The Latest Breach" is to kind of explain things in a way that other people can understand.

David Moulton: I love that. It's so difficult at times to avoid the jargon or the specific language of the industry. Even some of the FUD. Like let's just ramp up the fear because it does seem exciting and scary. It gets almost Hollywood. But to move away from that, and just the facts, talking about it in a way that's accessible, you know, the fact that you do that is awesome. I appreciate that. And I'm seeing more of that in our space which is encouraging where it's content that's accessible to everyone. I hope to do that on the show actually. So I'm aligned with you on a principle level. Today we're going to get in to the XDR landscape and in to your process on building waves. We've got a lot to talk about. So let's see where this conversation goes. Allie, what was the most impactful thing you've ever done in your career?

Allie Mellen: The most impactful thing that I've done in my life that furthered both my career and my life in general was to do a values exercise which I don't know if you're familiar with, but there's an exercise that Brene Brown has on her podcast and also her website. It's totally free. You can like download this PDF that has all of the different values that you could potentially have in it. And she walks you through the steps of determining what your values are. And I really needed this maybe like 10 years ago in my life. And I was listening to a podcast that she did and hearing about this values exercise. And at first I was like, "I don't really need this. Like I already know what my values are." And I spent like two seconds thinking about it and I was like, "I really value being nice." But as I went through the exercise what I realized is that was not one of my values at all and if anything that was kind of just a way to hide who I truly was and what I truly valued in life. And so I went through this exercise and realized, "Oh, my god. My values are not at all what I thought they were." Because I don't actually feel good when I'm being nice all the time which sounds kind of weird, but there are situations where I would much rather tell someone the truth than do something that's nice and feels good for me in the moment. And so going through this exercise I identified that my core values are growth, respect, trust, connection, and playfulness. And that last one is actually really important because I love to be playful with like my friends and everyone, to be honest. But trust and connection are really linked and have changed a lot of the dynamics of how I approach situations because I went from trying to say the thing that people wanted to hear to saying how I truly felt and that helped me to connect much deeper with people and to develop a much better form of trust with people. So everyone's values are different. There's no reason that like certain values are better than others. But for anybody who is kind of thinking to themselves about how they define themselves and how they want to approach that, I'd recommend doing the values exercise.

David Moulton: Allie, that's the dare to lead list of values from Brene. Right?

Allie Mellen: Yes. It is.

David Moulton: Yeah. And you said playful. You said growth. You said --

Allie Mellen: Trust. Connection. And respect.

David Moulton: Yeah. You remind me of a book that I read years ago, "Creativity, Inc." It is about Pixar. Great, great movie house. And they had this idea of asking for your honest opinion and it put people in to a moral position. You can either be honest or dishonest. There's kind of a black and white piece there. And Ed Catmull and his team came up with this idea of candor. Turning the candor up. Turn it up to 11 if you will, to quote yet another movie. And I like this idea that you could move your candor up or down and over the years I've done that because I thought that was being, you know, open. And I could hear things without hurting someone. And somebody talked about the difference between nice and kind and nice is what you were talking about and kind is telling you, "You do have spinach in your teeth." As opposed to being nice and just letting it go. You tell the truth. And I suppose that one's not one that has a ton of consequence. And, by the way, yes. Tell me if I have spinach in my teeth. But I think that's interesting that the most impactful thing that you've done for your career is to go look at your values and be introspective, learn a little bit about yourself. And maybe it's a little bit fun. I know that playfulness is so important to you. I think that sometimes doing things that are fun or silly just because they delight you makes your day better, makes your life better, makes the people around you maybe smile.

Allie Mellen: It also makes it a little bit more lighthearted because I think that like one of the challenges with trust and respect as core values is that can get very heavy and like honesty that can get very heavy, but if you have playfulness mixed in there and you can still have fun with it then it's, I don't know, that is the balance that I like to strike. [ Music ]

David Moulton: So let's shift gears a little bit from this larger Allie Mellen conversation and go a little bit more focused on your work there at Forrester. Talk to me about the most surprising aspect of your cybersecurity research especially as our industry has evolved.

Allie Mellen: The most surprising. So there's a couple of things that I cover. Right? As an analyst at Forrester. I focus on security operations. So that includes detection engineering, security analyst, the security analyst role, and from a technology perspective that's SIEM, SOAR, EDR, XDR and security analytics. I also cover nation state threats and AI and its use in security tools. As far as my research is concerned, I'd say there's a couple of things that are surprising. First in the job I feel very grateful that like kind of coming back to this values conversation my whole job is about being direct and honest and telling it how it is with the research. So that's really cool. And something that I think is very unique to the role that I have as a Forrester analyst. But what's most surprising from the research I'd say it's the -- it's something that I knew going into it, but I didn't realize how bad it was in the industry which is we really do spend so much time hyping up and talking about products when the biggest challenges in organizations are the people and the processes and the fact that the reality of the situation is the security practitioner role is very poorly defined. We don't really develop skills for security practitioners that are based on security as a practice. We expect practitioners to know how to use tools. And so there's a big divide in the actual process side and people side and how we develop those people and how we build processes within an organization that is ultimately supported by the technology. And I think that that's one of the biggest challenges in the industry and it's one of the reasons why I talk about analyst experience so much is we need to develop this as a discipline instead of just expecting people to be using tools.

David Moulton: Allie, the front side of my career I worked as a designer and first couple of years I thought if I could just master Photoshop I'm a designer. And I realized especially as I saw other tools coming in that that wasn't -- that wasn't going to cut it. I had to understand the fundamentals. I had to understand what I was solving for. It wasn't just to make something that was beautiful, but it was also functional, especially in the UX space. And what you're talking about I've seen over and over in professional -- professional roles where if you could just master the tools then you are a X. If you could just get to a level of proficiency on a set of tools, you're incredible in your role even if you don't understand those underlying principles and the foundational skills that would allow you to move from any tool set and any place to driving an outcome. What is it that fascinates this industry so much with tools? And how do we break away from that?

Allie Mellen: So it's a really good point, and I'm glad that you brought it up with that framing because the one thing that I do want to say is that like I'm also very cognizant of and recognize that sometimes you've just got to get the job that you were hired for done. And sometimes that is just using the tool. And so I want to give space for that because I think that that is very true. The part that I want to challenge in that is that you can get the job done by understanding the tool. You can't get the job done better just understanding the tool. That's where the people and the process side has to come in if you want to like actually improve the organization and improve the industry. So that's the first thing. On what you were saying as to why we're so fascinated with this, that's a -- that's a difficult question, but to be honest I think it ties back to we have if you think about it as far as tech is concerned first off I think across all of tech it tends to be people who like to focus on technology, don't necessarily want to be the businessperson in the room or to kind of be the one developing those relationships. There are exceptions, but especially with the roots of cybersecurity that's a lot of tech people who want to be in the tech, who want to be doing cool tech stuff. What that means, though, is that we're missing on some of the business side of how do we establish processes around this. What can we learn from other industries that have done this well? How can we operationalize this beyond just what the tech person is working on? And also how can we teach others? Because ultimately if you look at cybersecurity a lot of the talent that came up did it through trial and error that they did by themselves and not necessarily through going to school for it. And we even see this permeating the academic scene as well, to be honest, where even if you get a degree in cybersecurity or in my case a degree in computer engineering you're not prepared to walk into an enterprise and work in cybersecurity. The practices that you learn there are very academic and they are not built for the difficulties, the resource constraints, that you'll face within an organization or frankly the politics and the things that you have to navigate in business. So to me it's a combination of those factors that leads to just a difficulty getting to that next level of operationalizing something to be more effective than just that one person. And the other factor at play there is it's a really technical field. It is not easy to find these unicorns that not only understand the technology and understand what it is to be a practitioner, but also understand how to play the politics game and want to play the politics game in an organization. And so it's just rare to find that mix of a person.

David Moulton: So a couple weeks ago you and I sat down and recorded a podcast and the piece that stuck with me since then was you talked about your process of making a wave. It sparked a couple of questions. And for our audience could you give a quick recap of your process? Because I think that was -- that was the piece that surprised me and I think is really interesting that I'm not sure everyone knows about.

Allie Mellen: Yeah. Definitely. I certainly didn't know the full extent of it before I became an analyst three and a half years ago now. So the Forrester wave for those who are not familiar with it is basically our evaluative piece of research. Think it's the equivalent of the magic quadrant, but for Forrester. And we typically evaluate up to I think it's like 14 or 15 different vendors depending. And one of the things that I think makes Forrester unique in this process is that the person who leads the coverage is the one who leads the wave and does all the work for the wave. Now we of course have a managed center of excellence that makes sure that the methodology is consistent across waves and has us have a basically project manager that makes sure we follow that methodology. But when it comes to the person that you're going to talk to about implementing XDR and the person that you're going to talk to about the different options you have to buy XDR, that is the same person. That is me. And the same thing for the person who's going to be talking about security operations. So there's continuity there that I really value because I can talk about the process side and then I can say, "Okay, but this tool is or is not working for that process. And here's how we need to make changes to make sure that that's better." Now when it comes to the work behind the wave this is -- this is a three, two, sometimes five month process. We do the wave every two years typically. Sometimes we do it more frequently or less frequently depending on the market. But it is looking at up to 14 different vendors and measuring them against a series of criteria. Now over the course of those months we do a couple of different things. We get a questionnaire response from all of the vendors and that has a variety of different questions for each criteria. And the criteria can be up to like I think it's like 24 or something like that. And we measure vendors based on their strategy and then also their current offering. So we take a look at where's the product right now, where's the product going. And we score them based on that. And so we base it on the questionnaire is the first piece and then we do a typically two to three hour briefing and demo from the vendor to try and better understand, okay, what is the strategy for the future? And then let's actually get in to the product. Let's dig in to it, see what it's like, see what it's about. And then the last piece of this is we do a series of customer reference interviews. We try to do at least three per vendor because ultimately I don't necessarily know what it's like to work day in and day out in the technology, but I want to to be able to give better advice to our clients. And so I'll do 30 minute sessions with multiple customer references per vendor to make sure that I get a full perspective. And those are some of the most interesting and fruitful conversations because it's really fascinating if a customer reference really likes the product. It's also even more fascinating when the vendor gives us a customer reference and the customer hates the product or hates the vendor because that's where you get the real juicy stuff.

David Moulton: I'm sure. [ Music ]

Allie Mellen: The reality is we talk a lot in the cybersecurity industry about like, "Oh. What do customers need? What do we need to tell them?" They're so tuned in. Like CISOs are so tuned in to what's working in the industry, what isn't. Sometimes they just want a gut check on whether or not what they're seeing is the truth, but they're really tuned in and really aware of what's going on. And so I love having those conversations with CISOS and then especially with their teams who are actually using the tools because that's what I love is like is this actually making your lives easier as the user? Not just as the economic buyer. Or is this something that's like just a pain to use or a pain to work with the vendor in general? So that is a very helpful part of this. And then we spend several weeks evaluating everything that we've found. We also of course go online, look at the vendor's website, look at the different resources we have access to, do additional research. And we formulate a point of view on the vendors in the market. And now the cool thing about the wave is that it is relative to others in the evaluation. So when you get a wave score, whether it's a one, a three, or a five, that's dependent on is the vendor capability for that criteria is it on par with the market? Is it above par? Or is it below par compared to others in the market? And so everything is really based on where the market is currently at and where we expect that it should go. And the other thing that I really love about this process is we of course have the wave graphic which is based on the scores. All of the scores you can download an actual Excel spreadsheet and read in to what the scores were, what they mean, what the questions we asked to get to those scores and to get to those answers and insights. So you can get a really deep perspective of where we came at the evaluation from. And then of course we do a write up which kind of goes in to more of our point of view on where the vendor is at. So it's a very involved process, but it's also just you leave having such a deep understanding of the market.

David Moulton: Let me go a different direction. Is there anyone that you try to stay away from or that you prefer not to have to spend your time with during these research periods?

Allie Mellen: So for any research I'd say that the people that I don't like talking to are -- or that I struggle to get real value out of our conversations are the ones that are just trying to sell me something. Like to a certain extent I understand a vendor comes in. They want to have a -- they want to talk about how great their product is. But the challenge is that in a lot of those conversations they have first off think that they're the best which there's a lot of vendors in the industry that think they're the best. But in many cases they've lost sight of who the actual hero of the story is and it isn't the vendor. It is the user of the product. It is the CISO that they are providing and working for. And so I want to hear about that. I want to hear about what the customer problem is, why the product really solves this well, and how you've been able to support serious transformation in these organizations with what you've built. So that's kind of my biggest priority and my biggest challenge is like if I get in a room with someone who's telling me they have the best product in the world I already know we're going to fight and it's going to be I'm going to have to push them really hard to get to the root of what they're doing and whether it's actually helping customers.

David Moulton: It sounds like just like a good sock. You're looking for like that diverse number of points of view, different ways of seeing what the product does, and then looking for somebody that has that curiosity to go on that exploration and the research with you, not just a closed mind this is what the problem is, here's how to solve it, we're done.

Allie Mellen: Yeah because it's not that simple. Right?

David Moulton: Right.

Allie Mellen: There's a factor of respect here and respect for the people that have come before you, the people that have been working on this problem for a long time. And understanding that like you can have a really good solution to this problem. That doesn't make it the best in the world. But you've got to come back to the customer and the challenges the customer has.

David Moulton: So is there anything that you try to keep in mind through the entire process and are there any observations that you've made where vendors assume or get something wrong or right throughout that set of conversations and evaluation?

Allie Mellen: The thing that vendors get the most wrong in these evaluations is they approach it from the -- some approach it from the standpoint of what does Allie want to hear. And that actually plays in to the start of your question which was about what is something you keep in mind throughout this. The thing that I want to keep in mind throughout this is that I might not be right. And that's really important to me is I don't go in to this research with a point of view like these vendors need to fit in the box that I have created and then they're going to be the best. I go in to this with a perspective of I want this vendor to convince me that what they're doing is right for the customer. Maybe it's not something that I have ever considered as an option, but if they can convince me it's right for the customer that's differentiated. That's interesting. That's cool. And unfortunately a lot of the vendors that are part of this evaluation a lot of times they come in and they're like, "Well, we know Allie likes this. And we know Allie doesn't like this because she's written on this. So we're just going to say what we think she wants to hear." And the problem with that is that it often doesn't align to the point of view that the company has on the market. And that's the priority to me is like what's your point of view on where the market is going? What's your point of view on the solution and the way to get to the solution? I may not agree with it. I don't have to agree with it because I can tell you that not every client that I talk to, not every CISO that I talk to, agrees with my point of view. They go a different direction and then we have a discussion about why that worked or didn't work. And so when I think about these evaluations what I want is I want to see why what you're doing is important, who it's important to, and why it's different from everyone else. And that's not going to be something that I agree with 100% of the time and that's a good thing.

David Moulton: So, Allie, you talked about the most fruitful part of the conversation is talking to the customers. When you hear from those customers and they tell you what they want, they say very specifically, "I want a faster horse" and you're seeing that the market's got the Model T, right, how do you deal with that and how do you re-frame what they are saying they need when you're having a conversation with the vendor to understand does their vision or does that point of view align with what a customer actually needs if the customer's saying they desperately want something, but they're focused on the immediate solve, not necessarily the larger technology solve that's possible?

Allie Mellen: I love this question so much because it's like it comes up constantly, this idea of like oh let's just reinvent the wheel here to solve the customer problem, but we're so good at it that we're going to solve it in a different way kind of thing. Now with customers this is especially difficult right? Because I mean I was listening to a panel, a customer panel, for a detection and response vendor. And in one breath they were asked, "Okay. What do you want to see in the product? Like what would be really useful for you?" And they said, "We really want you to start doing configuration management and giving me visibility in to that because you do such a good job on the detection and response side it would be so useful if you could do a good job like that on the configuration management side." And then in the next breath they were asked by the moderator, "Okay. What do you not want us to do? Like what do you think is the thing that we need to be most careful of?" And they said, "Stay in your lane. Don't do something that you're not specialized in. We love what you do. We love what you're working on. Keep doing what you're good at." Those two things are completely at odds. Like they could not be more at odds. But the problem is is that like they're answering two different questions and they're giving honest answers to those two different questions. But they're not recognizing that sometimes a vendor will say, "Well, a customer said I had to do this so I'm going to do it." And we see that happening right now quite a bit with a lot of the changes that are happening in the sim market where many vendors are going, "Well, we're detection and response vendors. Our customers love us. But they want us to replace their sim. And so what should we do to do that? We should build a sim." And the customers are like, "Yay. You're going to replace my sim and you're going to do it better." But the biggest problem is how are they going to do it better? What are they going to do differently so they don't end up in the same issues that the sim has been in for so long? If we think about the sim market, look at a just based pricing as an example. There are so many vendors out there who have said, "We're going to get away from N just based pricing for the sim." Love that idea. That is a huge pain point for CISOs. But what ends up happening? They spend a couple of years burning investor money supporting a model like based on entities or pricing based on entities or some other model than N just. It doesn't work. It's not sustainable. And they default to N just based pricing after a couple of years. We even see this with hyper scalers. And to be honest if hyper scalers can't solve a data N just problem at scale and not defaulting to an N just based pricing model, why do we think that there's a different vendor who can? They're the ones actually supporting the infrastructure. They're the ones who could do this at the lowest cost. And so I always try to have this conversation with the customer when asking them, "Okay. You're trusting the vendor to do this. You want the vendor to do this. But why do you expect the outcome to be different? And how are you making sure that the outcome is going to be different?" And I do the same thing when I talk to any vendors. And that is one of the most difficult conversations to have because they want their immediate problem solved and they trust the vendor.

David Moulton: Great. So, Allie, what's next in terms of your research? Are there any new approaches or challenges that you're excited about?

Allie Mellen: So the sim market is kind of for anybody who's been tracking that it's kind of a bit of a dumpster fire right now. So that is the thing that I'm the most excited about and the most interested in. There's a lot of M and A happening. There's a lot of changes that are going on. And I really want to dig in to that because I get a lot of questions from clients on like, "What are my options right now? Like what am I going to do next or can I do next?" And so we're actively working on research in that area around data management and approaches to data management. There's a lot of changes that have happened in the broader data management space that we can apply to security. So between that and then also detection engineering is such an important topic to me because I think that this is one of the ways that we can develop practitioners better and actually give them a practice, those two factors are I think the most exciting things happening in security operations right now.

David Moulton: Allie, thanks so much for the conversation. This has been a blast. I really appreciate you sharing your insights and sort of a behind the scenes look at your process and your career and really going deep on what you care about.

Allie Mellen: Thank you so much for having me. This was really fun.

David Moulton: Before we wrap up, I want to invite you, the listener, to a special webinar that takes a closer look at the evolving XDR landscape. As cybersecurity threats grow more complex, extended detection and response has become essential for organizations to stay ahead. Join Josh Costa, director of product marketing at Palo Alto Networks, and today's guest Allie Mellen and myself for an insightful conversation on the latest developments in XDR. We get in to market analysis, share practical insights, and have a thoughtful conversation on the transition from EDR to XDR and what that means for your security strategy. I'll make sure there's a link in the show notes or you can search the Palo Alto Network site for the state of XDR featuring Forrester. That's it for today. If you like what you heard, please subscribe wherever you listen and leave us a review on Apple Podcast or Spotify. Your reviews and feedback really do help us understand what you want to hear about. If you want to reach out to me directly about the show, email me at threatvector@ paloaltonetworks.com. I want to thank our executive producer Michael Heller, our content and production teams which include Kenne Miller, Joe Bettencourt, and Virginia Tran. Elliot Peltzman edits the show and mixes the audio. We'll be back next week. Until then stay secure. Stay vigilant. Goodbye for now. [ Music ]