Anthropic’s court battles.

Dave Bittner: Hello, everyone, and welcome to "Caveat," N2K CyberWire's privacy, surveillance, law, and policy podcast. I'm Dave Bittner, and joining me is my co-host, Ben Yelin, from the University of Maryland Center for Cyber, Health, and Hazard Strategies. Hey there, Ben.

Ben Yelin: Hello, Dave.

Dave Bittner: On today's show, Ben discusses a second case decided in a dispute between the U.S. government and Anthropic. I've got a look at AI prompt histories as evidence. While this show covers legal topics, and Ben is a lawyer, the views expressed do not constitute legal advice. For official legal advice on any of the topics we cover, please contact your attorney.

Dave Bittner: All right, Ben, now, we've got some interesting things to cover here today. Why don't you kick things off for us here?

Ben Yelin: Yeah, so if you'll remember, a few weeks back, there was a case we discussed from the Federal District Court in Northern California about Anthropic's lawsuit against the federal government. As you'll recall, they lost their contract with the government. They were labeled a national security risk because of a dispute between Anthropic and the Department of War. The Department of War wanted them to do certain things, like mass surveillance and autonomous weapons. They said no. The Department of War said, all right, we're going to punish you, right? So we got that one case from the Northern District of California that we covered. I mentioned at the time there was a separate case in the D.C. Circuit Court of Appeals, and that has now been decided. If you'll recall, Anthropic won that first case we discussed. They lost this one, so we have a bit of a circuit split. Although the cases are not identical, I still think this is an issue that's going to need to be resolved, ultimately, by the Supreme Court.

Dave Bittner: All right. Well, tell us more.

Ben Yelin: So basically, the central question here is whether the Department of War, who originally had the contract with Anthropic -- and I can't believe I have to call it that. Can we do Defense still, or --

Dave Bittner: Oh, I still do Defense because Congress still technically named it "Defense."

Ben Yelin: That's true. All right. We'll do Department of Defense. The question is whether DoD concluded that Claude created a supply chain risk within the meaning of the federal statute. So the federal statute allows the executive branch to label a contractor a supply chain risk and would allow that executive branch to terminate the contract between whatever that contractor is and the government agency. According to a two-one decision here, the majority emphasized several facts: One is that Anthropic intentionally trains Claude to follow the company's normative values and policy judgments, which I think is roughly true if we listen to what Mr. Amodei says. Anthropic can modify future versions of Claude through model training and embedded restrictions. Earlier versions of Claude had refused government requests, including classified national security tasks and other inquiries, and there was a live dispute over Claude's use in a sensitive military operation overseas, creating concern that the model might not perform when military personnel expect it to. Based on all of those facts, the Court concluded that, at least as it related to the Department of Defense, they reasonably fear that Anthropic could influence Claude's operations in ways that might prevent the system from carrying out missions in support of our national security. Therefore, the Secretary -- in this case Secretary Hegseth's -- determination that we needed to discontinue our government's relationship with Anthropic was neither arbitrary nor unsupported by the record. Anthropic tried to argue that the statute was much narrower, that it should only apply to cybersecurity threats such as sabotage or espionage. The Court disagreed with that and said, if you read the statute broadly about supply chain risk, it has to do with, really, any scenario in which a supplier's ability to manipulate the operation of any information technology product can constitute a covered risk, so something that's going to present a risk to that federal government agency. They also held that there were no less-intrusive alternatives. I think Anthropic would have said there are ways that you could have gone about severing your relationship with us that wouldn't have been a sort of blanket ban on any contractors with the federal government or with the Department of Defense using Anthropic. That would have been more constitutionally acceptable. This Court didn't buy that. They basically said once Anthropic declared that these AI systems wouldn't be available for these requested tasks, the Court found it reasonable to conclude that narrower remedies would not adequately address the risk that was presented by that. They also -- Anthropic complained they didn't get proper due process. The Court disagreed with that. The key area of disagreement with the other court's decision, and I think this is where the Supreme Court is going to get involved, is Anthropic is claiming that it's being punished not just for a disagreement on policy, or some type of contractual dispute, but for the views that they expressed about AI safety and government regulation. That severing, maybe it makes sense in the narrow context to sever our relationship as it relates to autonomous weapons and mass surveillance, but for every other government function, they're not being punished because they're actually a supply chain risk. It's because they were critical of the government's use of AI, and the Court rejects that claim. They say that the Department acted because Anthropic refused to accept these contract terms. The Department thought those terms were essential, and really, you can't prove at all that this had to do with Anthropic's political views. And this is really the polar opposite from what the Federal District Court in Northern California said, which is they were being punished under the First Amendment for their speech. You know, I think we have this big disagreement here, and this is the type of case I think would be ripe for Supreme Court review.

Dave Bittner: You mentioned that the two cases aren't exactly parallel to each other. Are there key differences that will matter here?

Ben Yelin: Yeah, so the biggest one is that the D.C. Circuit Court is only upholding the Department of Defense's exclusion of Claude from the Department supply chain under the Federal Acquisition Supply Chain Security Act. The Northern California decision, which is actually a temporary restraining order, which was later made into an injunction, that was on the broader restrictions that applied government-wide. So in one sense, you could say the state of the law right now is we have 100% assurance that the Department of Defense is within its authority to exclude Claude from the Department's supply chain, so that includes all military contractors, anybody who contracts with DoD, but it's at least uncertain, at this point, how that applies to other government agencies, who are not the Department of Defense. I think Anthropic can count this as a loss as it relates to challenging the Department of Defense's policy. I think the result is undecided as it relates to all other government agencies. You know, you can get into these complicated territories where contractors might have work that implicates the Department of Defense but isn't directly with the Department of Defense. Maybe they have a large federal government contract, and they serve several agencies, and one of those agencies is the Department of Defense, and it's very unclear. I mean, I'd hate to be a compliance person at one of these contractors when you get, kind of, these conflicting views. Nevertheless, I mean, despite these differences, we still have this disagreement over constitutionality, which is very fundamental, and that's going to have to be resolved one way or another, and the only entity to resolve it probably is the United States Supreme Court.

Dave Bittner: Why do you say "probably?"

Ben Yelin: Well, so this was a three-judge panel of the D.C. Circuit Court of Appeals. It was two Trump judges in the majority, and the dissenting judge is one of the oldest judges on the court, a George H.W. Bush appointee. You could have the D.C. Circuit petition to rehear this en banc, which would mean that the entire D.C. Circuit Court of Appeals would hear this case. The breakdown of judges is more favorable, potentially, to Anthropic on the full Court of Appeals panel than it is for these particular judges, so it's possible that Anthropic moves for a rehearing en banc. If the judges on the D.C. Circuit decide to grant that, we could have another merits hearing, oral argument, and then another potential decision that may uphold what this three-judge panel said, or may go against this three-judge panel, so that's, kind of, the one possible step. Most requests for en banc hearings are denied, just because I think a circuit wants to respect what a three-judge panel has done. They don't want to second-guess all their decisions. It's kind of like, if you, as another judge, second-guess their decisions, they're going to end up second-guessing your decisions --

Dave Bittner: Right.

Ben Yelin: -- and that's just going to be a lot more work for the D.C. Circuit. So--

Dave Bittner: Kick it up to "The Supremes."

Ben Yelin: Yeah, I mean, that's really what they probably all want, is we know eventually that this is -- because we have a dispute among circuits, there's sort of an inevitability that the Supreme Court is going to have to get involved. So, like, why take this extra step to do this en banc hearing? There's, kind of, a judicial doctrine that addresses that question, that if something's so obviously on a glide path for the Supreme Court-- usually because there's a disagreement among federal circuit courts -- let's cut the crap. [laughter] Let's go straight to the Supreme Court. Let's get ourselves a resolution here.

Dave Bittner: Okay, and given the makeup of the Supreme Court, which is different from the D.C. en banc group, any tea-leaf reading?

Ben Yelin: It's really difficult to say with this Supreme Court. I mean, a lot of the Justices have been pretty deferential to national security in a lot of contexts. We haven't had a case that's directly like this with these nine justices. You know, we have a lot of Bush-era cases where certain policies that straddle the constitutional lines are related to things like indefinite detention or even mass surveillance. You know, you had a real split on the Court about people who were willing to make accommodations for national security, especially in a time of war, and Justices who were not able to do that, and so cases came out, kind of, 5-4, one way or the other, throughout the 2000s. We have a completely different court here. There are, I think, some of the conservative justices, particularly Justices Gorsuch and Barrett, who are a little more libertarian-"ish", civil libertarian-"ish," and, you know, might not automatically take the Department of Defense's side in this dispute, just because if they really feel that Anthropic is being unduly punished for expressing its First Amendment rights in the form of speech or expression, they might be willing to say that that's a constitutional violation, and you would expect that the free liberals would join them. So it's really unclear. You know, Alito and Thomas, the most conservative justices, have always been very deferential to the executive branch's invocation of a national security need, and so you, kind of, start with two votes in the bag there. Kavanaugh, kind of, seems like he would be on that train. I would suspect Chief Justice Roberts, based on his history as well. So yeah, you kind of have Gorsuch and Barrett as the swing votes here. I would definitely watch the oral argument, because there would be a level of suspense to it.

Dave Bittner: [Laughing] Right. You'd be on the edge of your seat with your popcorn?

Ben Yelin: Yeah, read those tea leaves.

Dave Bittner: That's right. That's right.

Ben Yelin: Was that in the form of a question, or was that an exclamation point?

Dave Bittner: Yeah, I guess this case has me wondering what this would be like if it were any other thing but AI? In other words, suppose this was just a big contracting company, like, let's do Deloitte or Accenture. You know, any of those -- I put air quotes around -- "the usual suspects." You know, the people who provide massive amounts of services to the federal government. What if instead of AI, they were providing consultants, experts on these particular things, and they said, "There are just some things we're not going to consult on because it's against our company's core values and beliefs," right? And then, the Department of Defense comes back and says, "Oh, okay, fine. Well, then you can't do any other government contracts."

Ben Yelin: I don't know how strong that metaphor can possibly hold, because it's not just, like, we're not going to provide a consulting service. It's operational support, so it'd be more like a weapons supplier being, like, "We are not going to allow our weapons to be dropped on Gaza, because we have a fundamental disagreement with the Israeli government's position."

Dave Bittner: Oh, right, right, so this is a weapons supplier trying to decide what targets are available or not.

Ben Yelin: Exactly. Now, you know, in that circumstance, you might say, yeah, of course, the Department of Defense should only purchase weapons from companies that are actually going to use them for DoD's purposes. But if there was a full federal government ban on that contractor, I think it would still implicate constitutional issues. Let's say it's a huge company, and only one of their services is the production of weapons. So, you know, they refuse to participate in some DoD operation. DoD says, sorry, we're terminating our contract with you, and then, also, you're a supply chain risk, so every other federal department is prohibited from doing work with you.

Dave Bittner: Right.

Ben Yelin: Again, that still seems to me to be overbroad because there's no -- at least to my eye, there's no non-speech justification for the exclusion. Especially if that contractor were to say, you know, we have no moral problems or qualms with any of the other work that the federal government is doing. We will provide weapons or whatever. In every other context, we will fulfill your work orders, except as it relates to this one contract. Then it's like speech really is the only reason why they're being punished in those circumstances, and, you know, courts hate that. They hate when they can see that somebody, some entity-- whether it's a person or a corporation -- is being disfavored because of what they said, or what their viewpoint is. You know, we saw that just this past week when President Trump tried to exclude CNN, MS NOW, and Politico from the White House, and those news sources immediately got an injunction because whatever any other government official says, Trump himself was very clear that they are being excluded because of what they've said about him, and courts hate that. They don't like the government punishing people or organizations based on speech, and I think that would certainly be a live consideration here.

Dave Bittner: Well, I always ask you, what's the timeline that we're on here? Is this next year, or could it happen quicker to make it to the Supreme Court?

Ben Yelin: It's always so frustrating. I think now that we have a circuit split here, I'm more convinced that this is going to make it to the Supreme Court. My guess is it would be for the next term, 2027, 2028. You know, the timeline is very frustrating, because, like, by the time we're in June 2028, when I suspect we could get this decision, you're already six months away from a new administration taking office, and so it's not exactly moot, but, like, it's just, we're going to allow whatever the status quo is, which for now seems to be Anthropic can be barred from DoD, and probably cannot be barred from other government agencies. That sort of status quo is just going to remain in place, and we're going to have that type of legal limbo unless there's some type of groundswell among Supreme Court justices to put this on a quicker time path. You know, first we have to get an appeal from Anthropic, which I would suspect. They might request that en banc hearing. You know, once we're through with that whole process, I'd be curious to see if the Supreme Court thought that this was an urgent enough matter that they'd fit it into the current term, which is about to begin, the October 2026 term. I kind of doubt they would, but I've been surprised in the past.

Dave Bittner: Well, and, I mean, there's the factor that, you know, 18 months is an eternity when it comes to the evolution of these AI models.

Ben Yelin: That's ultimately the bigger point here. I mean, we've talked for years about how the judicial system in our country, and the legal system generally, including legislatures, they move far more slowly than the technology. But, like, that, I feel like, understates the problem, given the rapid development of AI. I mean, especially when you talk about recursive generation, like, who the hell knows what kind of capabilities Anthropic and Claude are going to present to us in the spring of 2028?

Dave Bittner: Right, right.

Ben Yelin: If you believe some of their engineers, we could all be dead by then, but even if you don't, like, we're talking about major, major changes in capabilities that, is the case going to be moot at that point because we're litigating over the capabilities as they existed in February and March 2026? That would be a live question in this case.

Dave Bittner: Yeah, we're legislating prop planes, and it's the jet age.

Ben Yelin: Yeah, I mean, that's kind of how it works.

Dave Bittner: Yeah.

Ben Yelin: It's just -- it's frustrating from that perspective. I don't know what a good solution is. Like, the legal process is the legal process. It takes a long time for legitimate reasons because both parties have to brief and, you know, you have to have the chance for amicus briefs, and each side has to be prepared for litigation. You want to put your best foot forward, but those timelines can be problematic when we're talking about rapidly changing technology, and if there was ever a rapidly changing technology, this is it. I mean, we used to talk about it generally on cybersecurity matters, but this is a big advance even on that timeline.

Dave Bittner: All right. Well, we will have a link to that story in the show notes. Let's take a quick break here to hear from our show sponsors. We'll be right back after these messages. [ Music ]

Dave Bittner: And we are back. Ben, my story this week comes from -- it's actually a blog post from Halkyn Security, which is a U.K. company.

Ben Yelin: Shout out to our U.K. fans.

Dave Bittner: There you go. They have put up this blog post. It's titled, "Reading Prompt History as Evidence," and it got me thinking about how, you know, we've over the past, I don't know, a couple of decades now -- since the beginning of search engines -- we hear these cases where there's been a crime committed. Someone is up to no good. Someone, you know, dispatches their spouse. Police later go look at their Google search history, and it says, like, "How to bury a body," right?

Ben Yelin: Right.

Dave Bittner: Yeah, you know?

Ben Yelin: "What's the best way to cover up the crime that I just committed?"

Dave Bittner: Right, right. Exactly, and so that's been a thing for a while now, but now we have prompt histories, the history of what people -- their interaction history with their AI assistants and the legal ramifications of that. I think similar to a Google search, a lot can be revealed, but also, similar to a Google search, a prompt history doesn't necessarily prove that a particular person put in that prompt, right? It could be anybody.

Ben Yelin: Sure can, yep.

Dave Bittner: Especially, you know, go to your local public library and do the prompt there. 

Ben Yelin: Usually people aren't dumb -- you know, usually people aren't dumb enough to put in these prompts with their own IP address, but sometimes you have the clever criminal who's willing to go to that public Wi-Fi. It's not always foolproof. I've seen enough Law and Order episodes where they pull surveillance cameras on who was using the public library computers at that particular time?

Dave Bittner: Right, yes.

Ben Yelin: You know, at least at least you're putting in an effort.

Dave Bittner: So let's start with some basics here, someone's prompt history. Let's say I'm suspected of being up to no good, and the police say they want to take a look at my prompt history as part of their investigation. What do they have to do to get at that?

Ben Yelin: So I wish I could say there's a clear answer there. At least in the United States, this is an unsettled area of the law. I think the question courts would have to consider is, do prompts count as routing information, the type of thing that would be on the outside of an envelope in a physical world? Or is it content, which would be on the inside of an envelope? If it's content, which I generally suspect it is, then you need a warrant, so a judicial authorization supported by probable cause. If it's non-content, you might just need a subpoena. The standard is lower to obtain a subpoena. It's generally just reasonable suspicion as opposed to probable cause. Usually, you think this would be cut and dry, like the contents of an email, for example. We know, according to court precedent, that requires a warrant, but an HTML or, you know, like, a search bar, that has been held to just be routing information. But as you know, links can contain, like, half of the news articles just in the name of the link, so it's not always cut and dry. I would suspect that courts would see this as content, and that would require a probable cause determination. So these companies would not be able to divulge what these prompts were from the prompt logs unless they were authorized to do so through a warrant issued by a neutral magistrate.

Dave Bittner: One of the things that they highlight in this blog post, they say, "Investigators should distinguish between the request, the AI's response, and subsequent human actions. An abandoned request generally shouldn't be treated as evidence of misconduct unless making the request itself violated policy." I'm curious about that "abandoned request" element.

Ben Yelin: Yeah, I mean, I don't know. To me, it's, like, thinking from a legal perspective, if that's evidence that the person had a guilty conscience -- or that they at least had the intention to do some type of search that would aid them in the commission of a crime -- that, to me, should be fair game evidence because that gets to one's criminal state of mind. Like, I don't think there should be a distinction whether you pressed "Enter" to actually submit the prompt. Maybe that makes sense from a technical perspective, but just to me, from a legal perspective, you know, I guess the difference is reasonable expectation of privacy. If you put that prompt into the ether, you pretty clearly lose it, and if you don't finish typing your criminal confession [chuckling] and just, kind of, keep it in that text box, then maybe you still do retain that reasonable expectation of privacy, but, you know, it is kind of unclear to me how that would work in practice.

Dave Bittner: Yeah, they say, "AI-generated responses aren't evidence of a user's intent. Models can produce material that doesn't accurately reflect what a person intended or ultimately did." So in my mind, this is like me saying to the LLM, or let's just extend it and say me saying to the hitman that I ran into in an alley. "You know, I'm really frustrated with my wife." And the AI responds, "It sound like you want to kill your wife. "

Ben Yelin: That sounds more like Clippy than an LLM, but yeah.

Dave Bittner: Right, right [laughter]. "It looks to me like you're trying to bury a body. Would you like" --

Ben Yelin: "I understand what you're trying to do now?"

Dave Bittner: "Would you like to shop for shovels?" Yeah, so that's an interesting wrinkle, I guess. But again, if the AI can connect the dots, certainly the prosecutors can, right?

Ben Yelin: Yeah, I mean, I think it's, ultimately, you're not using the generated responses as evidence. You might use the generated responses to get what you really want, which is the prompt. Maybe there's a technical method where you could identify the prompt by the AI-generated response. But yeah, I think it's perfectly fair to say that AI-generated responses don't reflect an actual user's intent because I think your example is a perfect one. It could be something completely innocuous. Like, "I'm very frustrated with my wife." It could mean, "I want therapy." It could mean, "I just want a sounding board, somebody to hear my problems," and if AI takes it in a criminal direction, then the person who put in the prompt should not be held criminally liable.

Dave Bittner: Right. Right. Yeah, and well, I mean, that leads me to another thing that they say here. They say, "Proportionality is critical. Investigators should resist collecting an entire AI history simply because it's available."

Ben Yelin: Yeah, I mean, and that's like textbook Fourth Amendment stuff. You should, if you're seeking to collect something like this and you need to obtain a warrant because it's content, then you have to have some type of particularity requirement met, so the specific prompt you want seized. How you identify what that prompt is could be difficult. You know, they talk in this article about how prompt histories are stored in different places. I am certainly no expert on where this stuff is stored. It's all a series of tubes to me. [laughter]

Dave Bittner: Right.

Ben Yelin: But, you know, from a legal perspective, I think that is critical that you can't take somebody's entire history with Claude or ChatGPT. You'd have to limit it to a single prompt or a series of prompts that implicates criminal activity. Otherwise, you get close to what our Constitution abhors, which is general warrants, where it's, like, yeah, let's just search this guy's full ChatGPT file and see if we can find any evidence of criminal activity. That is an offensive concept to our Founding Fathers and to our current jurists who think that would just be an overbroad search, so I think proportionality is absolutely critical. You know, it's certainly reflected in the structure of our legal system, so I definitely agree with that.

Dave Bittner: Is it fair to assume that we're going to start to see this issue pop up in courts, that people are going to challenge the inclusion of chat histories and how they've been acquired?

Ben Yelin: Yes, because I think it's a really good form of evidence, and we have seen a limited number of cases where it's been -- chat prompts to LLMs have been admitted as evidence. I think it's kind of the same jurisprudence that you see with reverse keyword search warrants, where, yeah, it's really relevant who searched what, when, and generally, the constitutionality of that has been upheld. I don't know how that would go post Chatrie. That's a whole other conversation, but I could see, like, a reverse prompt warrant where you could go to Anthropic and say, can you give me all of the prompts during X time period in which somebody requested advice on how to bury a body or how to cover up a crime? And I think that would be a very valuable piece of evidence. We've seen chat prompts introduced in evidence for other purposes. In civil cases, we've certainly seen it, civil cases against these companies. They've included evidence of what the user originally submitted as their prompt and what the allegedly negligent response was, so, yeah, these are going to be a major part of court proceedings. It's also an opportunity for people who are familiar with these technical details to be expert witnesses. You know, having an expertise in the type of stuff that I don't understand, like what prompt logs are and where they're stored, that's going to be a very valuable skill. You know, courts might have one or two people that they rely on for these types of cases, and so that kind of technical expertise could be very useful. You could be the expert witness on that Law and Order episode.

Dave Bittner: [Laughter] Right. "Bom, bom."

Ben Yelin: Yeah, you thought you were just a tech nerd, but your expertise has come in handy.

Dave Bittner: Right, absolutely. All right. Well, we will have a link to that story in the show notes, and of course, we'd love to hear from you. If there's something you'd like us to discuss on the show, you can email us. It's caveat@n2k.com. [ Music ]

Dave Bittner: And that's "Caveat" brought to you by N2K CyberWire. We'd love to know what you think of this podcast. Your feedback ensures we deliver the insights that keep you a step ahead in the rapidly changing world of cybersecurity. If you like our show, please share a rating and review in your favorite podcast app. Please also fill out the survey in the show notes or send an email to caveat@n2k.com. This episode is produced by Liz Stokes. Our Executive Producer is Jennifer Eiben. The show is mixed by Tré Hester. Peter Kilpe is our Publisher. I'm Dave Bittner.

Ben Yelin: And I'm Ben Yelin.

Dave Bittner: Thanks for listening.