How to buy security products.
Rick Howard: When Cyber Security began as an idea, sometime in the mid 1990s, meaning that leadership in Government, academia and the commercial world started to realize that they needed to formally do something to protect their digital assets, because you know this Internet thing might really take off. Who knew? They started hiring Info-sect teams to manage it. Back then we all bought hardware security tools to perform a single function. You would buy a firewall to block unwanted incoming traffic, you'd buy an intrusion detection system to alert you if bad guys were knocking on your door. Every security tool we had was a hardware device that did one thing, and we would stack them up on each other in a giant rack and insert the entire contraption between the organizations Internet connection and the internal network. Very quickly, we all started lovingly referring to this configuration as "the security stack".
Rick Howard: As time went on, vendors started inventing different kinds of security tools, that were a hybrid of software and hardware, and not designed to be deployed at the Internet boundary. You know, things like In-point protection tools. But as a community we still labeled this collection of security tools as "the security stack" even though it wasn't a stack anymore. And that didn't change as we started adding multiple Internet connections to our environments for redundancy purposes and it didn't change when we started adding multiple physical locations to our networks, like the headquarters in Boston, and a sales office in New York City and a Data Center in Northern Virginia. "The security stack" was the collection of security tools that the Info-sect and the networking teams used to secure the organization in cyberspace, and that did not change as we started moving workloads to the Cloud either. The only thing that really changed was the number of tools we used and the complexity involved in managing it all.
Rick Howard: The point of all this is that one of the key functions of the Security Executive ishow to manage the life-cycle of the organization's "security stack". Everything from deciding what to include in the "security stack" and how to pay for it all. And that could be everything from commercial products to open source tools to home-grown software that you build yourself, to managing and maintaining the deployed tool set in production, to deciding when a currently deployed tool is at end of life, and orchestrating the process to get rid of or replace it. This is a difficult job. Let's find out how to do it. My name is Rick Howard. You are listening to CSO Perspectives, my podcast about the ideas, strategies and technologies that senior security executives wrestle with on a daily basis. Joining me at the Hash table today are two Cyber Security Executives who have been in the industry for years, and have seen all sides of this managing the "security stack" process.
Rick Howard: Helen Patton is currently the Advisory CISO for Duo, an authentication tool withing the CISO constellation of security products. She is also the Chairman of the Cyber Security Canon project, and just last month announced the author and inductees into the Hall of Fame for the 2021 season. Nick Gilbert is the Cherokee Nation Businesses CISO, an old friend of mine and a regular here at the Hash Table. I asked both of them about their process of determining which tools to include in their "security stack".
Helen Patton: For me, the decision to buy something really aligns to the security strategy that I have got going on, and the security strategy is formulated with a few things in mind. Thinking about where the business is headed and what the business wants to go is going to be another thing and if I don't have a security solution that's going to support that business objection, I am going to go shopping. When I was the CISO at Ohio State and we were already on this path of "distance education and virtually learning" kinds of things, the reality was that my security stack wasn't great at things like authentication or monitoring of on-line teaching and learning environments. Real-time monitoring kinds of things, not just config. If the CEO makes the decision to put everything in the Cloud, you have got to make sure your "security stack" can cover that, I'm going to have to probably go shopping, because you don't go from on-prem today, Cloud tomorrow. There's this hybrid piece and most tools are not going to handle hybrid from end-to-end. So, at some point I may be able to de-com some security thing as I no longer needed, but in the interim I've got to have all of it, so I'm going to go shopping there.
Nikk Gilbert: It is really about assessing the risk of the organization first. When I say risk I don't mean every single risk factor within the organization, it's a quick risk assessment. What's the intellectual property look like? What's the risk tolerance of the corporation based on the vertical and the compliance regulatory requirements, based on who are the threat actors and what types of threats are you facing? Doing that rudimentary gap analysis is really an extremely complicated process. Getting a general sense about the business and the potential risk that could cause material impact is always a good idea. But the very next thing is this strangely parallel world of compliance, where most security practitioners believe that only small pieces of it actually help improve your security posture. But because we are talking about national and international law here, you can't afford not to pay attention to it.
Helen Patton: Do I have a compliance requirement that I am currently not filling? That is going to be my step one. Am I complying with the law? If I am not and I don't have a tool that is going to satisfy the compliance requirement, then I am going to go shopping for a tool.
Nikk Gilbert: Imagine you are in a company, a medium large size organization. You have your CISCO "security stack", and it is probably going to do 80% of your security. Let us take the same Company and pull it out of the manufacturing vertical and stick it in the financial services vertical. You are going to need tools that CISCO doesn't have. If you're in financial services and you have a trading floor, man, that's some complexity, but you've got some money and you're going to be complaint or you're not going to do business. Once you understand the risk environment environments at a high level and you understand the compliance environment, how do you decide which tools to buy or even to consider? I mean, one conservative estimate from Ned Miller at McAfee back in 2018, pegged the number of existing commercial security vendors to just over 1200. 1200? That's a lot.
Rick Howard: Most security practitioners, after doing some preliminary research on their own, will reluctantly turn to one of the security analyst firms like Gartner or Forrester. With their deep analysis of industry markets and their production of business intelligence products, like the Gartner Magic Quadrant or the Forrester Wave, that ranks various vendor products against their competitors. I say reluctantly, because there is a perception in the security and IT community that vendors have to pay the Gartners and the Forresters of the world to even be considered in their evaluation. The worry from the practitioner side is that if a vendor is paying the analyst firm, then the vendor should expect a positive review. That said, most security organizations don't have the resources to keep track of 1200 security vendors every year. So at the very least, the Gartner Magic Quadrant and the Forrester Wave can provide a CISO with a first cut of products to consider. I asked Helen how she went through this process.
Helen Patton: The Google. [LAUGHS] It usually starts with the "Google", but yes, I am going to go to the Gartners and the Forresters, and I'm going to look again to see what industry analysts are talking about? I might check out CyberWire, if you have had them on, to talk about their product. I might look at an on-line magazine, any sort of industry product comparisons. There is a pay-to-play aspect sometimes that you just need to know exists. And what I get out of the Gartners and the Forresters is just an understanding of how the analysts in the industry are thinking about the strengths and weaknesses of the product grouping. Again, I may not go to the individual vendor for that explicit thing. For me, whether the vendor is good or not, will depend on how they work in my environment, which is not going to be the same for everybody. But at least, getting the base level of understanding of what this group of products is meant to do, how do I ask questions about it, what questions should be asking, those kinds of things, that is what I go to the Gartners and the Forresters for.
Nikk Gilbert: When I think about a technology, I go out to the Gartner Magic Quadrant first. I'll look at the top three in the upper right, and I'll look at maybe a couple of challengers or visionaries, to see if they're interesting from a cost perspective. I find that sometimes by including a challenger into my suggested "stack", that you can sometimes find a really cool tool at a really good price.
Nikk Gilbert: Okay, I get it is pay-to-play, but nine out of ten times, that's where I go. That's where a lot of organizations go. Gartner's a two billion dollar company, and a lot of CIOs listen to Gartner, so it is an easy way for me to get quick buy in. Every boss I have had for the last ten years has been a Gartner customer, and if I wanted to do something or if I wanted to sell something internally I'd be like, okay Gartner, good check. "Hey boss, we need this new tool, we need this new technology, we want to implement this process or procedure". As soon as I say Gartner, "Okay, good, good" you know. If I was sitting as CISCO in a start-up right now, I probably wouldn't be able to afford Gartner. You can't call an analyst, you can't compare X to X, you don't have an account manager who digs up stuff for you. You have got to do it all yourself. I go out to Google, Google Endpoint protection and there's Magic Quadrant. That's only part of the story. The rest of the story is having someone do the research for you and having Gartner is like having several full time employees on staff. That's the sale of Gartner.
Nikk Gilbert: From a start-up perspective, you cannot afford it. You're going to talk to your friends and your peers, that's where the real value is at, if you will. You are not going to have a Gartner subscription. Okay, you can do Magic Quadrant from Google.
Rick Howard: So far we have a quick risk assessment, a deeper look at compliance requirements and a touch of the Gartner's and Forrester's to narrow down the list of potential tools. I would call all of this the preliminary work, but it starts to get serious when the Security Executives start to reach out to their friends and colleagues to see if anybody has anything good or bad to say about the products they are considering for their "security stack"
Helen Patton: This is where my networking comes into play. When I was the CISO in higher-ed, the higher ed CISOs, to the extent we could without violating NDAs, we would talk about the pros and cons of the products they were using, what worked in their environment, what didn't. Often our environments were similar, so it became a starting point. Separate from that, I am part of other networking groups of security practitioners in it that are cross industry and international as well, so I have got these sounding boards where I can go, "Hey I am thinking about this class of products. What are you all using in this space? What do you think about it?" Or if I've got to the point where I am considering a vendor that I might want to bring in for a POC, I might explicitly ask that question. But it's that peer group referral is probably more impactful to me than the others. I start with word of mouth. I call it background research, but it is really talking to my peers, seeing what they've got with a bias towards people whose environments are somewhat like mine.
Nikk Gilbert: The real money is the peers. When I talk to other CISOs, call up any one of my CISO friends and say "Hey, have you deployed this technology, or have you tried to do this in your environment? What do you think? And then we will have a good conversation and there is more value in that conversation than, I think, anything else, besides having done it already yourself, which, obviously you are in some type of quantum loop there, but there doesn't a day go by that I don't ask somebody something. I am constantly soliciting information from whether it's my own, you know, company or other people who I know from other companies, or friends, there's so much value in hearing the opinion of others on these types of things. And people that are hungry and absorb all this different data, different ways and they may have a completely unique opinion or perspective on something than you do. And you get them in the conversation, you're like, "Man I never even thought about that that way, that's really valuable." Gartner is one thing but that peer/peer interaction is absolute. It's not even gold, it is like platinum. It's higher than that. It's "Unobtainium".
Rick Howard: The next step in the process is to conduct a proof of value, or a proof of concept exercise, where you actually bring in the vendor tool and set it up in a test environment to see how it will fit into your current "security stack".
Nikk Gilbert: Once you get to a point where you think you've found the right tool, your peers have said, "Hey this worked for me", doesn't mean it is going to work for you. Everyone has a different environment. And then you bring in the couple of vendors that you want to try out and have them do their proof of values. That's where you really see them shine, because the proof of value team is supposed to be their "A" team. That is the best of the best. At least in my perspective, it should be. And that will give you an indication of how the service is going to be, how well the product truly works.
Helen Patton: Some of the proven concept is, how flexible and understanding is the vendor going to be to my environment and the way that we have to work, and how understanding are they of that, in addition to the technicality of the product? My experience has also been, if you've got two products that are similar, the differentiating factor is their professional services, and their sales relationships, more than the technical product space.
Rick Howard: Before you do a proof of value exercise, both Helen and Nikk recommend that you spend some time developing the criteria you will use to choose the ultimate winner.
Nikk Gilbert: Hopefully you have a security architect. If not, you build it yourself. It is an Excel sheet. It probably consists of, depending on the complexity of the project, anywhere from 20 to 50 rows, and 10 to 20 columns. You've got your vendors across the top, you've got your criteria down the side. What's your SLA and what's your OLAs? How does it feel? It's not just a bunch of "does X=2" or whatever. It is much deeper than that, because, as you've alluded to earlier, corporate culture is extremely important. It depends upon what level of buy-in you have as well. As a security practitioner if you don't have the buy-in of the CEO than you might as well just stay home. You just have staff work through it and fill it out, does it meet this requirement? Does it do this? Does it do that? How does it do this? At the end of the day you have a very very valuable document.
Rick Howard: One way to do these exercises is in a shoot-out type of scenario, where you are in real-time evaluating each vendor against each other. But Helen doesn't like this kind of competitive set-up.
Helen Patton: The challenge for my internal team, is having two vendors in the space at the same time, it's hard to juggle for them from a time and resource perspective. From their perspective, it is easier to do one after the other, but there's always trade-offs. I do not like doing a shoot-out, because they don't expect vendors to have exactly the same product. There is often overlap, it's a Venn diagram and I want to know what the outer edges of their product set is as well.
Rick Howard: But Helen is also quick to point out that the judgment criteria is way more than just technical. She is looking to build a long-term relationship with this vendor.
Helen Patton: I want a security vendor who is going to make me successful as a CISO. So if that means they are going to establish relationships with my CIO or my CFO or my CEO, or whoever, in addition to me, or with me, then that's awesome. I am absolutely going to pay attention to that. What I do not want is a vendor who, because they know the CEO or the CIO, they go to that person and say, "Tell your CISO that they've got to buy this product". That is going to kill the deal right there. In the context of higher-ed, whatever product I purchased had to cover about 100,000 people. There was no product that was $20,000 or less. It was always a million dollars here, and a million dollars there, and it always caused sticker shock with my CFO and CEO in broad. So it was really helpful to have a vendor who would be able to come in and help me make that case as to why that's reasonable. I am looking for that in a partnership as well as the technical side, for sure.
Rick Howard: And Nikk points out that one of the things you get with a long term partner is a commitment to keep his operational team up-to-date on how to use all of the tools the vendor is adding to the "security stack"
Nikk Gilbert: I went to my vendors and I said, "Hey we know things are bad, you guys are a partner, business is not moving as fast as we would like it to be, so I want you to bring your people in, and I want you to train my people. I want you to spend 40-80 hours with these guys and train them how to use these tools to the maximum capability". Not only did it strengthen the relationship between myself and my staff and the vendor, but it also gave my people the knowledge that they needed, and they were actually trained on the tool.
Rick Howard: Which brings us to the last phase of the "security stack" life cycle. DE-commissioning a tool that is at end-of-life, no longer necessary, or is not working the way you think it should. And with most things in cybersecurity that is a lot easier said than done.
Helen Patton: Well, it depends on how insidiously embedded the product is in the tech stack. I did have somewhere the CIO had made a decision to go and buy something that was almost duplicative of something that I bought on the security side, and then the question was, "Why are we paying additional for this product?" And so, I had to make a call, "Is this the hill I am going to die on?" and say no, the security difference is such that we are going to stick with this or not. In one case I said "No, we are going to sunset this product." So it was basically a reverse of everything we did when we rolled it out. We told people that we were going to be sunsetting it, we gave them time to migrate to the other solution, and we did it all in time before our contract ended with the vendor, and we shut it down.
Rick Howard: Like Helen said, the process of getting rid of a "security stack" tool is similar to how we got it in there in the first place.
Nikk Gilbert: If I have a current vendor in the "security stack" and it's at the end of their enterprise license agreement, I am going to probably go through a very similar process as I did in the beginning. Do the whole Gartner selection criteria, proof of value, everything that we talked about, if they've kept things up to speed. If you remember Norton Anti-virus just at the turn of the millennium, they were the most awesome company ever. Norton was winning all the awards, and then they got the biggest Government contract they ever had and they changed the semantic and the whole company went "to poor performance" is the best way I can put it. You could not get anybody on the phone, the product didn't work, you caused brute screens of death in Windows 2000. It was just a nightmare. I would probably just jettison somebody if they did not keep their technology up to speed, if they didn't continue their RND and continue to buy companies and evolve their products and show me cool things and keep me informed on the road maps and things like that, then they're probably going to be on the way out.
Nikk Gilbert: And that's just going to be very simple. You're done, we're done, we will turn your product off and we will move on. Yes, I know that in itself is complex. What kind of prerequisites, what are the intricacies of the software that are interacting with your own software, and on and on and on? If you are not happy with the tool, get it out of the environment.
Rick Howard: I reminded Nikk that in my experience removing tech from a "security stack" is almost as hard as putting new tech into it.
Nikk Gilbert: I do not disagree. Everything we do is a major league project. Speaking from my chair, as an Enterprise level CISO, it is all major league. If you are in a smaller company, it may be easier, but part of me is CISO. You are going to have some good tools, you are going to have some bad tools. We've all had them, and as long as you approach it in what I like to say, a sustainable and reputable fashion, then you are going to be successful. I would be devoid of emotion making a decision like this. When you are talking about technology protection from a "security stack" you may have a sales guy you don't like, but you are not going to cut off your nose to spite your face kind of thing. If I am going to remove a product, it's got to be either just end-of-life or they have just completely missed the opportunity that they had to continue to compete with the other people in their space.
Nikk Gilbert: It is very expensive if, let's say, you are going to replace your entire stack. Product X for product Y, and product Y is $50,000 cheaper. That $50,000 is probably not going to be a savings, because you are going to pay that just to yank the old stuff out. And you've got to include that in your business case, you absolutely have to include that in your business case, it is imperative that's identified. You may find, at the end of the day, that you want to keep it, but when I think about removing your product, I am not removing it from a cost perspective, I am removing it, because it no longer does what I need it to do.
Rick Howard: As we were working on this episode, I had an epiphany. It's this idea that managing the "security stack" for whatever organization that you work in, is a key and essential part of the Security Executives function, and we hardly ever talk about it as a thing. All we talk about tools and strategy and risk and resilience, but we hardly ever pause to consider that the security stack is the essential organizational framework within the people process technology triangle. In the security world the "security stack" is the technology side of that triangle and we spend a lot of resources in terms of money spent in human toil to maintain it. And that's a rap. As always, if you agree or disagree with anything I have said, or anything our guests have said, hit me up on LinkedIn or Twitter and we can continue the conversation there.
Rick Howard: Next week is the last episode of the season and we have a great one planned for you. We will be talking about how all of us keep up in this ever-changing world of Cybersecurity. In other words, how do you stay current. You don't want to miss that. The CyberWire or CSO Perspectives is edited by John Petrik and executive produced by Peter Kilpe. Our theme song is by Blue Dot sessions, remix by the insanely talented Eliott Peltzman, who also does the shows mixing, sound design and original score. And I am Rick Howard. Thanks for listening.
