Is AI security actually a physical problem?

Mark Houpt: You get an informational vulnerability that will then allow you to pivot to a low vulnerability and warm your way in, whereas a human is not capable, or it takes a long time, and again it's highly detectable when a human's doing that. With AI, it's not. So we have to go back and rethink. [ Music ]

Johnny Hand: Welcome to AI Security Brief, where we're unpacking emerging AI threats, vulnerability research, and the strategic decisions security leaders are making right now. I'm Johnny Hand.

Dustin Childs: And I'm Dustin Childs. Today, we're talking all things data center security. With so many of the conversations around AI security, we often overlook the fact that these AI systems exist in the physical world. Our guest today is Mark Houpt, CISO for DataBank, and he is one of the rare CISOs who led early AI innovation projects, building an AI security framework from scratch.

Johnny Hand: Definitely. Mark really dives into the many different types of risks around AI data centers, not just the physical access controls, but the impact of modern AI attacks and how nation-state actors are planning insider personnel to disrupt operations. So it's not just about the traditional cyber attack anymore. There's a broader spectrum of threats to defend against. And, Dustin, I'm sure that you were geeking out on the threat awareness conversation.

Dustin Childs: Absolutely, I was geeking out about several parts of this conversation. So let's go ahead and just dive right into it.

Johnny Hand: Mark, welcome to AI Security Brief. We're very excited to have you here.

Mark Houpt: I am very excited to be here, especially since we were intending to do this one other time and flight delays got in the way.

Dustin Childs: As they often do.

Mark Houpt: Mm-Hmm.

Johnny Hand: So, Mark, for listeners who may not know you, or as they're going to look you up, can you briefly describe your role at DataBank and then, you know, also how it connects to AI and data center operations, which is what we want to talk to today.

Mark Houpt: Yeah, absolutely. So, you know, when you hear Chief Information Security Officer, there's a role description that comes to mind. And that's typically inside of an enterprise type of organization. That individual is responsible for the cybersecurity of the organization. Within the data center role, we have that function that exists for our corporate side, but we also have additional security functions that apply for our customer side of things. So my role encompasses three different disciplines. One is that cybersecurity and security architecture. The second one is in regulatory compliance as it applies to information security. And the third is physical security of the physical buildings, as well as our corporate facilities.

Johnny Hand: Got you. Now, I know that you've been in security for decades, for many years.

Mark Houpt: Yes.

Johnny Hand: How have you seen the rise of AI just change the way that you look at your security of your environment or securing your environment?

Mark Houpt: Well, it's interesting. You know, we go through these cycles. So it's hard to believe back in the 1990s, when all this got started, or when I got started, when you guys got started, that we really didn't have security departments in our corporate environments. And if they were, they were rent-a-cops quite frequently to secure large buildings. But that said, what I've seen -- and the point of that is that I've seen that there is a foundational understanding of security. And what we keep redoing is reinventing security with new titles and new names, but it's really the foundational security that we've always done. Role-based access control. We could call that by a hundred different names. We can call it zero trust. We could call it whatever you want to call it. But it's still limited access based upon your role, based upon what you're trusted to access. So there's -- so what we're doing with AI is very similar in my opinion. Now, there's always nuances to it. But I have two things that I tell people in my team and elsewhere. "What's old is new again" is a phrase that we can really apply to security. And I try to tell people that because they have to understand that we can apply the same methodologies and technologies and thought processes to what we're doing with AI into our current environment. And the other thing I like to say is, look, I've been doing this long enough -- there's a lot of people that have been doing this long enough that we can evolve those basic control sets into how we operate. So, physical security, for example. It hasn't changed for AI. We need to protect the environment. We need to -- we need to put fences around. We need to have security guards. We need to check people's IDs when they come in the door. We need to zone things off and put multi-layer security in place. We need to put multi-factor authentication in place. All these things are things we've done for years, but we might do it differently. And our threat landscape is probably the biggest thing that's changed. So, five years ago -- so I've been at DataBank for 11 and a half years as CISO, so I've seen a lot of changes happen. Five years ago, our biggest threat was people trying to get into the facility to update their servers because, "Oh, by the way, I left my badge in the office and they won't let me in the office anymore because of COVID." And now we're dealing with situations where we still don't want people to get into the data centers, and we want to protect them, but we have different influencers that are trying to do that. Whether that be nation states because we have new wars and new situations going on, or we have what I call the NIMBY situation. Again, not something new, but not in my backyard. The new part of not in my backyard is, "Hey, I don't want a data center in my backyard because of what people have said about data centers." But the whole NIMBY conversation, you know, farmers don't want new industry in their backyard because they're taking away fields. Well, now the new industry happens to be data centers.

Johnny Hand: You know, that's interesting, Mark, because when I talk to people and, of course, I talk about AI security, they're obviously thinking about cyber. But you've said that as a co-location provider, physical security can be the most important thing for customers. Why is that?

Mark Houpt: Well, a couple of different reasons why. One is the chipsets that are used inside of these compute devices that actually can, you know, conduct the transactions for the AI, conduct the compute for the AI, are actually much more valuable than standard compute. You know, the GPUs that are placed in there, there's a huge demand for it right now because there's so much AI being put in. It is the thing, if you will, to do. Now, the other thing that's being layered on recently in the past few months, past six months or so, is the national security issue, where some of the AI that's being run on these GPUs in particular in the US and in other nations as well, they have an aspect where it's so advanced that one country doesn't want the other country to get it, even though it's not a classified product. And that's an interesting challenge for the three of us, having been former military. We clearly understand this whole, you know, confidential secret, you know, top secret with, you know, SCI potentially. And even though these GPUs are not classified, there are now executive orders in place as of two weeks ago that says you cannot send these things overseas. You cannot use the AI that's applied to them, not even overseas, but with individuals who are not US citizens. And so those challenges are what arise. So there's lots of different aspects of that physical security piece that, right inside the data center, is a concern, not to mention the activists that are outside that are wanting to come in and damage and destroy those things because they believe that AI is running our lives.

Johnny Hand: Yeah. So you're concerned about nation-state actors, really resonates with me as a ZDI [inaudible 00:08:58] threat person. Specifically going back to Stuxnet, which had an insider component, where I believe it was a Dutch scientist who was visiting an Iranian facility, loaded a USB. How are you operationalizing that threat model at a co-location facility to combat against that insider threat, whether it be an intentional nation-state actor or a bumbling custodian, for example, who could be equally as threatening in the right circumstance?

Mark Houpt: Yeah. So first of all, the old methods of, you know, if you don't have -- if you don't need to have access to a data hall, then you don't get it. Or if you need to go in there -- if you're the custodian and you need to go in there, you're escorted. So two-person integrity are a couple -- couple of ways to do it. But, you know, in general, there's a shared responsibility because, you know, inside these data halls, the companies that are contracting for the services from a data center provider like DataBank, they have a responsibility because they can allow their people to go in. So what we need to do is we need to have a shared responsibility that who we are hiring is really who we believe that we are hiring. And there is a huge situation going on right now that I've been briefed on, and I'm sure you have as well by national law enforcement and even local law enforcement, or at least, state-level law enforcement, says, hey, there are certain countries out there that -- and you can go on the news and you can look them up yourself -- where people are creating laptop farms and then having their people hired to work with companies. And these laptops literally are, you know, having keys move and mouse move from electronic format, and they're really doing nothing, even though they are getting paid for doing nothing. And unfortunately, we have to look at people as potential threats more so than we ever have before. Stealing intellectual property of not only DataBank but also our customers, and having an impact upon the systems that maintain the AI systems is where we're having the challenge.

Johnny Hand: I'm curious. I want to switch perspectives a little bit. Mark, so we've talked a lot about the true physical security aspect of protecting data centers, but, you know, the physical security component of that is around AI-based data centers. How are you leveraging AI technology in the data centers to also perform security and to protect those data centers?

Mark Houpt: Yeah, I was hoping you'd get to this question. So, without giving too much away, I will say that you could use AI technology on anything from drones to robo dogs. So, you know, way out here, bleeding-edge type thing. But AI technology is inside of your camera systems and inside of your badge readers and inside of your biometric readers and things like that. And so, putting aside the bleeding-edge technology, let's focus on the reality of most scenarios. What we're doing is we're using that technology to reduce -- in some cases, reduce the number of security guard rounds that need to be done, so we can focus our human guards at the front gates and monitoring systems. So, like the F-35 analogy, they become weapon systems operators. Not to say that the security is weapon systems. I'm using that as the analogy, okay? But these devices, you know, with the follow me function, you know, somebody's on CCTV. They identify who they are. They identify because they put their badge up against a reader when they walked into the gate that identified their name and their access authority. It grabbed their picture on camera, followed them to wherever they go. And oh, by the way, these people are -- you know, you are maybe hitting every badge reader along the door. You know, we have people do that for the fun of it. And so what we're doing is we're using that technology to help weed out the things that we need to look at and focus on the things that we do need to look at. We're also using it to identify when people are playing around, and they shouldn't be. You know, like I said, sometimes people will walk down the hallway that's outside the data hall, and they'll just hit every reader they want -- you know, they see for the fun of it. Well, that registers a deny. So our systems will go, "Hey, we've got three denies." You know, kind of like what we used to do with Active Directory. You get three denies. You get five denies in a certain period of time. We lock you out. Well, in the previous generation of access control systems, we couldn't necessarily do that on physical security access control systems. Well, now not only can you do that, but you could tie it to the CCTV and see that someone's not following the path that they should be following to get to their data hall. So now we start asking them questions, "Why did you go the long way to get to where you needed to go?" Things like that are starting to be utilized, you know, to track -- or more behavioral tracking type of scenarios. And every iteration, we're building on that. We're building on it. We're building on it. It's like, you know, we all know. True AI is three years old at best. So every iteration gets better, and every iteration gets a little smarter. We inject new things into it. We're also using it for training. Using AI for physical security training. We're able to, in some of our CCTV, we're able to put ghosts out there. So we could detect whether our security guards are actually monitoring or not. We can inject multiple failed attempts into the system so that our security guards, are they monitoring the alarms that are up on the wall? You know, things like that. But we're building over and over again.

Johnny Hand: So when you think about, like, what's new in tech, I think we often get, like, kind of tempted, if you will, to throw away our old security playbooks or address it like it's a new thing. So what are some foundational principles that you think still apply even in the age of AI?

Mark Houpt: Compartmentalization is probably one of the biggest ones. You know, we've always -- we've always said, hey, you know, in physical security in particular, you should only be able to go where you're authorized to go. But I think we got -- or, you know, not just in my organization but in lots of organizations, we got to the point where we were stretching the boundaries of that. And I think what AI has done is it's caused us to readdress that, refocus that, bring the pendulum back towards the middle. You know, the risk on a GPU being taken out of the facility versus a regular CPU chip or a disc has caused us to rethink that compartmentalization to the point of maybe it's not just the data hall anymore.

Johnny Hand: So if I'm a technology or security leader just starting to lead AI projects, obviously, compartmentalization, you've said, is a foundational practice. What are a couple other foundational practices or mindsets that you'd insist on from day one?

Mark Houpt: Well, don't forget -- if you're doing code development, don't forget just good security hygiene practices. You know, even -- you know, with the new AI that's come out in the past couple of months, where they're attacking multiple layers, you know, 10 layers deep, then, you know, you've got to think about all the vulnerabilities that you've been willing to allow before because you had other mitigating circumstances in front of it. So we now -- you know, one of the things that we're doing inside of DataBank is we're going back, and we're looking at we've been really good at applying patches for critical and high and even medium level stuff. Not so great about doing informational and low things, but we've already seen attack attempts using AI that will drive towards those smaller things, and you get an informational vulnerability that will then allow you to pivot to a low vulnerability and warm your way in, whereas a human is not capable, or it takes a long time and it's highly detectable when a human is doing that. With AI, it's not. So we have to go back and rethink. So those are some of the things that we have to rethink because AI can do things faster than we can do. We get an exploit. Somebody says, "Go attack this in a lab." Boom. Now, go attack it in real life, and it's done.

Johnny Hand: And, Mark, we always ask every guest one last question, and so I know that we've talked a lot today, talked a lot about kind of foundational AI, how it's being used to secure the data center, but also the impact of, you know, physical security around the data center. Is there something that we missed that we should cover?

Mark Houpt: Yeah, maybe. The one thing that -- the one thing that I've been cautioning my people on is to not jump on the bleeding-edge technologies. Or if you do, do so in a very controlled environment. Because AI right now is so new, and people are throwing things out there that these products themselves have vulnerabilities in them that we don't need to be bringing into a data center, or we don't need to be bringing into an enterprise environment. Sometimes they're cool things like, say, a robo dog or a device that will go around and do the rounds of the security guard, or even a drone. But we have to make sure these are controlled, and they are actually appropriate for our environments. And I think that's the part that some companies might be missing, is their rush to deploy is not considering the long-term impact of what that deployment will be inside of their environment and the risks and vulnerabilities that will bring into their door, almost like a Trojan horse, a literal Trojan horse, not a worm or a cyber Trojan horse, but a literal Trojan horse. And so we have to be careful of those things. I think that's something we haven't talked about that people need to watch out for.

Johnny Hand: Yeah, I think that's sage advice, and I'll say as a security leader, the amount of marketing calls I've gotten around drone and, you know, robo security for years has been pretty insightful. But I think it's that same way that we don't want to jump into the early technology, especially at an enterprise level. It's fun to test and evaluate, but definitely not -- well, Mark, thank you so much for joining us on the AI Security Brief. I know Dustin and I have gotten a lot out of this. We really appreciate the conversation, and we look forward to chatting with you again.

Mark Houpt: Absolutely. Same here. [ Music ]

Johnny Hand: And that's a wrap for today's episode of AI Security Brief. A huge thank you to Mark Houpt for joining us all the way from Alaska.

Dustin Childs: And Mark covered something a lot of people overlook when they talk about AI security, and that's the fact that it's fundamentally a physical problem. If someone has access to your device physically, it's no longer your device.

Johnny Hand: Yeah, and we know that the threat landscape has changed. Now, we see nation-state actors, activists, and social media all being used as proxies.

Dustin Childs: Yeah, Mark's point about shared responsibility between co-location providers and their customers is something that every security leader in that model needs to hear. Insider threat is not new. It's just that the stakes just got higher.

Johnny Hand: Yeah, and what stuck with me was the framing that Mark did of what's old is new. So concepts like compartmentalization, two-person integrity, and really good patching discipline never actually went away. I think AI just raised the stakes of getting it wrong.

Dustin Childs: Yeah, his closing warning about rushing bleeding-edge physical security tech into production without understanding what you're bringing into your environment was the note I really wanted to end on because you could potentially be bringing in a literal Trojan horse and not a worm.

Johnny Hand: Yeah, I agree. And we'd like to thank Mark for coming on. If you've enjoyed this conversation, subscribe wherever you get your podcast and check out our show notes for links to connect with Mark and DataBank.

Dustin Childs: AI Security Brief is mixed and produced by Elliott Peltzman, with original music by Amneajynx. Our executive producer is Jennifer Eiben, with content strategy by Ma'ayan Plaut and Melanie Gallant. Additional production help by Liz Stokes. Video editing by Sarelle Joppy and Brigitte Criqui-Wild.

Johnny Hand: Thanks so much for listening, and we'll see you next time on AI Security Brief. [ Music ]