When AI serves up trouble.

Dave Bittner: Hello everyone and welcome to "Caveat," the CyberWire's privacy, surveillance, law, and policy podcast. I'm Dave Bittner and joining me is my cohost, Ben Yelin from the University of Maryland's Center for Health and Homeland Security. Hey there, Ben.

Ben Yelin: Hello, Dave.

Dave Bittner: On today's show Ben has the story of a judge rejecting a claim that chatbots have First Amendment free speech rights. I've got a closer look at AI liability. While this show covers legal topics and Ben is a lawyer, the views expressed do not constitute legal advice. For official legal advice on any of the topics we cover, please contact your attorney. All right, Ben, we've got some interesting things to talk about here this week. Why don't you kick things off for us?

Ben Yelin: So, this is a fascinating story. It starts out, though, with a tragedy. So, there was a 14-year-old boy by the name of Sewell Seltzer III, or Setzer III, rather. And he committed suicide last February. Right before he had committed suicide, an artificial intelligence chatbot encouraged him to, quote, "Come home to me as soon as possible."

Dave Bittner: I remember this story.

Ben Yelin: Yeah, I think we had referenced it in an earlier episode.

Dave Bittner: Yeah.

Ben Yelin: And it is really tragic. The chatbot was created by a company called "Character AI" And they were kind of semi-acquired by Google. So, Google licensed characters, technology, and hired its cofounders in a $2.7 billion deal. So, the family of Sewell Setzer is suing both Character AI and Google and trying to hold them responsible for the fact that the son committed suicide after communicating with this chatbot. Which brings up a really interesting question because these high powered companies are looking for defenses for this claim. They don't want to be held responsible. That's going to entail significant monetary damages. So, one of their defenses, among many, is that the chatbot's words are constitutionally protected free speech. And this went in front of a federal judge in the U.S. District Court for the Middle District of Florida. A judge by the name of Anne Conway. And she held that at this point in the litigation she is not prepared to hold that chatbots have First Amendment rights. I think this leaves open the question not only in the future of this litigation, but for other litigation going forward. I think this is a very unsettled question. You can see her really wrestling with it and the opinion itself. I think she's kind of unclear on whether to hold that these chatbots have First Amendment rights and I think there's -- you could really go both ways on it. So, there just is a couple of preliminary matters. There's a question of whether this is actually speech or just expression. Because it's unclear that anybody is speaking here. There have been previous cases where the Supreme Court and other courts have held that algorithmic decisions, even those produced through the use of artificial intelligence are a form of expression by the company that makes them. So, for example, if you set up an algorithm that filters out certain types of, quote, "hateful contents" that is an expressive action. And the Supreme Court held that in a couple of Supreme Court cases that we've talked about over the past couple of years that the use of AI, depending on the circumstances can be expressive in nature. What's happening here is different. This isn't something that can be easily traceable to a decision by an actual human being at Charcter.AI or Google. Because this is just the chatbot itself, the LLM going rogue and using what it's been trained on to freelance and have a conversation with this individual and it's just unclear how you could trace that and hold responsible the people who work at these companies that simply planted the seeds that allowed this chatbot to grow into existence. And that's essentially what the judge here is saying. There have been where non-humans have been held to have free expression rights. One of the most interesting examples she uses is NPCs, non-player characters in video games --

Dave Bittner: Yeah.

Ben Yelin: Who -- and this is like a whole thing in video games and I don't really understand. I'm not much of a gamer.

Dave Bittner: Okay.

Ben Yelin: I believe, you might be more of a -- a little bit more of a gamer than I am.

Dave Bittner: Marginally so. Most -- I'd say most of my modern gaming comes from being in proximity of my youngest son. So --

Ben Yelin: Yeah. I think we're in the same boat on that. Basically, non-player characters can have expressive speech, even if it's in the context of the game, it's the game makers themselves who had some say in what these non-player characters say. Even if it's offensive, even if it could be legally actionable.

Dave Bittner: Okay.

Ben Yelin: So, that it causes some type of legally recognizable harm. I think the issue here, at least the way the judge sees is it, is we're too far attenuated from any decision than the actual human being made. They set up this chatbot. The chatbot is going to learn. It's going to flourish as it has communications with tons of individuals. It's going to develop its own personality of sort. And that's ultimately the responsibility, and this is getting super meta of the chatbot itself and not the people who brought it into existence. So, that's kind of the preliminary holding here. This is a very complex case and there are a ton of other relevant legal questions that have nothing to do with free speech, so things like jurisdiction, which I will not bore you with here. Other preliminary questions about who the correct parties are, etcetera. So, this was just a preliminary motion. I think once we get into fact finding and discovery, there's a chance that if you can actually trace the personality of this chatbot to some decision that was made by a human being in developing it, then perhaps those human being can be held liable. But at this point, the defendants in this case haven't asserted that type of -- haven't properly asserted -- or the plaintiffs haven't properly asserted that type of evidence. So, at least for now, chatbots in this circumstance do not have First Amendment rights. I think no matter what happens, throughout the rest of this case and through the appeals process, this is a question that courts are going to have to answer. Because we've talked about all of these stories of especially kids relying on AI companions. Which is really sad. I mean it's because there's a lack of real human companionship that they're just hanging on to these AI systems, but as these systems suggest things, like suicide, or like committing violence or things of a sexual nature, there's going to be an effort to hold those chatbots legally liable and we're going to need some type of foundational decision on whether they can be liable and under what circumstances. So, I thought this was a super interesting story and kind of curious to hear your thoughts on it.

Dave Bittner: So, I'm reminded of a couple things that I think could be interesting to interject here. One is do you remember years ago there was the whole issue with there was a monkey that took a selfie.

Ben Yelin: Yes.

Dave Bittner: And there was a case over whether or not basically the monkey had copyright protection.

Ben Yelin: I do remember that. Yup.

Dave Bittner: And they ruled that monkeys don't have copyright protection because they're monkeys. So --

Ben Yelin: Could have made that a one sentence decision. They don't have First Amendment rights because they're monkeys.

Dave Bittner: Well, and monkeys have -- yeah, so you have to be human to have copyright protection. But time passes and we get these AI systems, these LLMs. And so, the question then was if an AI LLM creates an image, is that image eligible for copyright? And my understanding is that so far they've said no that it's not. Because it's not generated by a human.

Ben Yelin: Right. Right, which is kind of is unfortunate for the people who own that intellectual property because it is a reproduction of that person's property, they could suffer a financial loss because somebody is using their intellectual property without paying for it and there isn't really a chance for any type of recourse. Same thing is happening here. Like, I understand the plight of the parents of this kid just thinking, like, there's a very clear reason why this happened to my son, it was this chatbot. And somebody needs to be held responsible for this. And whether it's about the money or not, you understand the impulse of just being like, this terrible thing happened, this chatbot is instructing my son in so many words to commit suicide, somebody has to be held accountable and I just think courts are kind of handcuffed right now and don't really know what to do to answer this question. I think they're doing their best to wresting with it. With the limited case law that they have, I mean in this decision, they're citing previous cases that aren't really parallel to the facts of this case. I think they are citing concurrences in Supreme Court cases, so opinions that aren't the controlling opinions. I think they're rasping at straws to try to try and figure out an answer to this question. And I think it's going to continue to vex them. One way the companies have reacted, which I think is appropriate is trying to develop AI software suitable for children and specifically targeted for children. It will have greater guardrails on it where, like, its key instruction is never ever tell a child to commit suicide under any circumstances or even suggest it under any circumstances.

Dave Bittner: Or how about just anyone [laughing]?

Ben Yelin: I mean that would be great. I think it's especially important for children who are not only impressionable, but and this is depressing to think about, I think many kids, especially younger kids don't realize that they're talking to something that doesn't have feelings or sentimentality or loyalty, and they don't understand that this is just -- this is just a computer. Because the computer is so good at mimicking human emotions. So, presumably, adults are better able to understand this. They understand the risks of it. So, they if they sign up for, kind of, a wild west AI software where anything goes, they at least have some type of expectation. I think you can't really say that for kids. So, I know Google has recently announced its setting up an AI powered Gemini app that's going to be available specifically for children. There have been conferences with developers. They mentioned one in this article where Google is putting out a suite of other new AI products that are geared towards kids. So, there's definitely a market developing for this where your -- things are more tightly managed in the context of things that are going to be available to children.

Dave Bittner: Yeah.

Ben Yelin: And I think that's the best way of attacking this problem now until we figure out what the key legal principle is here and whether these companies can be held accountable.

Dave Bittner: I'm curious in the copyright issue. To what degree does a human's -- I say, 'monkeying' with --

Ben Yelin: So to speak.

Dave Bittner: Yeah. And artificially generate. So, I say to the AI, you know, "Generate a picture of a monkey taking a selfie." All right. The AI generates the image of the monkey taking the selfie. Now, currently that image is not eligible for copyright protection because it is generated by the AI. How much messing with that image do I have to do as a human for it to then be eligible for me to copyright it?

Ben Yelin: I think that's a completely unanswered question. That's a very good question. Then it's like, okay, can AI developers have it both ways? Where they're not held accountable for stealing somebody's copyrighted material, but then they're suing other people for copyright infringement on images they created. I mean, I think you at least as a matter of fairness and equity, I don't' think you should really go -- well, I think there should be one standard as it relates to AI. Like, either it is sentient and human, in which case it should know better than using copyrighted materials as part of their algorithms and it has the capability to sue others for mooching off whatever the AI generated image is, or it's not. And the reverse is true in both cases, if that makes sense.

Dave Bittner: Yeah, I mean, I think I might be coming at this from a slightly different direction than you are. Because what I'm thinking of is as the human takes the AI generated image, like I want to be able to copyright that image for my own uses. And not allow anyone else to use it. So, in other words, I'm not interested in the AI companies' rights to that image.

Ben Yelin: Right.

Dave Bittner: I'm interested in my ability to sell that image and protect it. And so, if I have the AI generate an image of, let's just say a smiley face, you know, which we agree currently is not eligible for copyright protection, if I color correct it, if I alter a pixel or two, that's my question. So, is this one of those things where, you know, they get of the legal loosey goosey language that like, significant alterations must take place to --

Ben Yelin: I mean it's going to end up like that and we're not going to know, like it's just going to be all the case law that develops for us to figure out what a significant alteration is. I mean, you could apply that not just for images but for writing as well. Like, all right, I wrote the first three chapters of a novel myself. And I copy and pasted those first three chapters, put them into an LLM and said, "Finish this book for me." Whose intellectual property is that? Can you claim, given the fact that you've trained the AI on your own writing that the entire book that's produced should belong to you, in terms of copyright protection, or does the AI get to develop its own level of protection? How much can you claim in those circumstances, I don't know.

Dave Bittner: Yeah.

Ben Yelin: Because I just don't think we've settled these questions at all. Certainly, our policy makers at the federal level haven't come down one way or another, so it's just up to courts to really try to interpret these really complicated cases.

Dave Bittner: Yeah, and it's such early days.

Ben Yelin: We are in the extreme early days of this and I think the more I read this opinion from the district court judge in Florida, it's more her saying, like, I just don't know what the hell I'm supposed to do here. Like, there's never really been any guidance on this. There isn't proper case law, where I can just have my law clerks go and find what the relevant precedent is. Like, I'm flying in the dark here. And as a district court judge, I don't feel comfortable setting a new precedent that's going to impacts at least as persuasive authority, judges across the country. And I think this is a decision born out of judicial restraint, a recognition of I don't have the answers to the questions at this point, the legal status quo is that chatbots created through artificial intelligence when they are used just in LLMs and there's no expressive messaging on behalf of the creators is not constitutionally protected speech. I think that's going to be the status quo until somebody really works through this, goes through discovery and figures out exactly what nexus you need to have between a human creation and the output of the LLM in order to establish free speech rights. And I just don't think we're anywhere close to being able to properly establish that. But I think it could be in this case. That's why I'm interested in following it going forward to figure out how this judge and future judges are going to adjudicate cases like this.

Dave Bittner: Let me throw another hypothetical at you just for fun. So, continuing our theme of Sivians, suppose --

Ben Yelin: Can't I just say I don't deal in hypotheticals?

Dave Bittner: No! [lauging]

Ben Yelin: All right, bring it on.

Dave Bittner: I'm the host of this show! [laughing] So, suppose, so I'm extending off of the thing that a monkey cannot have copyright protection over a photo that the monkey takes. Right?

Ben Yelin: Yeah.

Dave Bittner: All right, suppose that I train a chimpanzee to fire a gun and kill someone.

Ben Yelin: Oh god.

Dave Bittner: So, I say I am so tired of Ben Yelin, I'm going to train this chimpanzee that when Ben comes in the room the chimpanzee is going to pick up a pistol and aim it at Ben and unload the magazine into Ben and that will be that.

Ben Yelin: See, I think that's a more common circumstance. I think you would be held liable under this hypothetical. Because we've seen that with liability related to things like dog bites.

Dave Bittner: Okay, okay, yeah, yeah, yeah.

Ben Yelin: Someone has trained a dog to be aggressive and violent. And if you have enough in discovery indicating that the person has trained the dog to attack certain people, then you can impute responsibility on the trainer.

Dave Bittner: Yeah.

Ben Yelin: And I think that would be implacable here.

Dave Bittner: Right, but does that extend to the training of AI models? If I've trained an AI model to the point where it's recommending suicide --

Ben Yelin: I think so, I don't think it's a perfectly parallel argument.

Dave Bittner: No.

Ben Yelin: Again, this is getting really complicated.

Dave Bittner: Right.

Ben Yelin: I don't think it's a perfect analogy because you could train an LLM, for example, and maybe this is what companies should do from now on, that the first mention of the word "suicide," or anything else, like the entire system shuts down. It's reported to whatever national authority and law enforcement gets involved. And that's probably what these companies should do in those circumstances. What's happening here though is it's gotten so far attenuated from any instructions that the developers have given it. Like, this kid had a, you know, multi-month conversation with the AI. And the AI is constantly learning and figuring out how to react as if the AI was human to the prompts in this conversation. You can't really do that with a dog. Like, it would be the equivalent of somebody not just training and instructing their dog to bit somebody, but like cultivating a relationship with a dog over a period of years, where they sort of have an implicit understanding with one another. Then it becomes a much more difficult legal case. I have trouble just thinking about this because it's so complicated. I don't know about you, it's just, it's a very vexing problem.

Dave Bittner: Right, but that's what makes it so much fun.

Ben Yelin: Yeah, exactly.

Dave Bittner: All right, I'll tell you what, we're going to take a quick break here to hear a message from our sponsor. And then we will continue our conversation. We'll be right back. [ Music ] All right, we are back and my story this week, actually, just kind of rolls right into what we were just talking about and this is story from Wired and it's titled "Who's to Bame when AI Agents Screw Up?" And it's kind of extending the conversation that we've had because I don't know if you know, if you've heard, Ben, but the new hotness when it comes to AI are AI agents which are the LLMs extended to be able to do stuff for you. So, I think probably a common example is going to your AI agent and saying, "Plan and book my vacation." Right? And it will go off and it will know. And it knows what types of things you like to do, and where you like to travel and what you don't like.

Ben Yelin: Another blow to the travel agent business right here.

Dave Bittner: Right, yeah. That's exactly what they need. So, it comes back to you and it says, "Good news, I've booked you air travel. I've booked you hotel stays. I've booked you rental cars. I've booked you tours." All that kind of stuff on your behalf, I did all this. So, away we go. Your vacation is completely planned. And that's the dream, right, that's the thing we aspire to.

Ben Yelin: I'm sure it just always happens that way, right Dave?

Dave Bittner: So, this article is about what happens when you give an AI agentic authority and it screws up. So, let's start with the example that I was just talking about. I ask it to book my vacation, I get to the airport and the flight that I was booked on doesn't exist.

Ben Yelin: Yup.

Dave Bittner: Right? The AI hallucinated, completely created a flight that doesn't exist. It's got a flight number. It's got, you know, I know people who are listening are going to say, "Wel, then how did it pay for it?" But let's just stay with me.

Ben Yelin: Yeah, and there's actually a real world example that we talked about on this podcast. Remember, the Air Canada chatbot scenario where the Air Canada's chat hallucinated a bereavement policy that didn't actually exist?

Dave Bittner: Oh, right, right, right.

Ben Yelin: Yeah, and they were like, "Here's your reinbursement as part of our bereavement policy."

Dave Bittner: Right.

Ben Yelin: Which, of course, Air Canada didn't actually have and it was a complete hallucination.

Dave Bittner: Right.

Ben Yelin: Talk about a troubling use case for hallucinations. I think agents is certainly up there when we think about the downfalls of LLMs and hallucinations.

Dave Bittner: Well, and there's another one sort of coming from the other direction. I remember there was someone who was using a chatbot, like a customer service sales chatbot on a car dealership website and got the chatbot to agree to legally binding language that would sell them Chevy Tahoe for a dollar. Right?

Ben Yelin: Woops!

Dave Bittner: Yeah, the guy was like -- but you know, he was like you agree that this is a contract, that this is legally binding, that you are representing the company. And the chatbot was like, "Yes, I agree. I represent -- " you know, it was that sort of thing. I don't think the guy ended up with a Chevy Tahoe for a dollar, but it's an interesting use case. It's an interesting not so hypothetical that actually happened. So, I'm curious on your take here. You know, looking through this article and the key points they make here, we know these AI agents are going to mess things up and what does that mean for what we consider liability to be these days, is this something where you get yourself -- insurance companies start selling AI agent liability policies?

Ben Yelin: I think they're going to have to, otherwise the companies are going to be sued. And this, again, is very similar to our last story here, but if you have an agent that messes up a significant number of times, and if you can establish damages and they give a really fun example in the story of somebody ordering onion rings but it is interpreted by the agent as ordering extra onions. It just gets lost in translation.

Dave Bittner: Right.

Ben Yelin: If you have, like a 10% rate of these types of errors happening, then I think liability has to be imputed on the agent. That should be the incentive to get these companies to refine their systems so that when people are using them, they could have more confidence that they're not going to get these rather absurd results. What the companies can do is try and contract that away. Now, that's not going to be a full proof solution but they might have a provision saying by agreeing to use this agent, you are acknowledging that this has a significant, there's a significant rate of failure or of mistakes through the use of this agent chatbot.

Dave Bittner: Right.

Ben Yelin: And you are either willing to not hold us liable, or you agree to whatever type of arbitration is going to be favorable to the company. Meaning, maybe somebody that a company is picked can adjudicate these cases to try and come up with an equitable solution.

Dave Bittner: Yeah.

Ben Yelin: I think that's probably the next step of what's going to happen. You know, it seems like a good use for our legal system to catch these types of valid legal harms that emanate from the use of agents so that the developers of the agents have more of an incentive to clean up their products. That's true for everything. I mean there's a reason that we have strict product liability. It's that if you have a product that's defective for too many users, it's not financially feasible for every individual user to instigate a lawsuit against the company and try and go through discovery and figure out at which point in the manufacturing process was one individual employee negligent that allowed this to happen. Products when they're put on the market are expected to work. Because we're going to be relying on them. And I think eventually we're going to get to a place where that's completely true of these AI agents. If they're widely in use, then there's going to be an expectation just like the producers of any other product that this is a product that works consistently and isn't causing harm.

Dave Bittner: So, let me extend my metaphor to the breaking point which is --

Ben Yelin: Here we go again!

Dave Bittner: I'm off to do, it's my favorite thing. Let's say I hire a travel agent, right, or let's say I hire a travel agency, so I'm hiring a company. They assign a travel agent to me. That ravel agent makes a colossal blunder. Same thing we envisioned before, right, the travel agent sends me information about my flight. Turns out the flight doesn't exist. Now, my normal next step would be to sue the travel agency, right, for screwing up my vacation, wasting my money. It seems to me like in today's world, the AI equivalent of that is the travel agency would have in their contract a line that says, "Sometimes travel agents make mistakes."

Ben Yelin: We make mistakes.

Dave Bittner: Or hallucinate. So, use our travel agents at your own risk. Now, a travel agency who did that wouldn't stay in business very long. And I think people would probably still perhaps successfully sue them. You think?

Ben Yelin: Yeah. There's a limit to how much you can contract away. Like, that's so haphazard that this continues to happen, the contract isn't going to completely get you out of it. And this is true for all different types of contracts. If it's legally recognizable risk, that's a normal risk attendant to that type of business, you can contract that away. But if you develop a chairlift at a ski resort where one out of every six chairs, or whatever, falls off and sends somebody to their death, then no matter how many times you get the skier to sign the release form, like that's going to be an unacceptable practice.

Dave Bittner: Yeah.

Ben Yelin: So, if we're relying on AI agents to perform more complex tasks that we're relying on to live our everyday lives, I think even with that type of contract language, or mandatory arbitration, courts are still going to be willing to hold those AI agents and vicariously their developers to account in those circumstances.

Dave Bittner: So, would you say that we are on an inevitable collision course between the companies who are making and implementing these AI agents and our legal system to work this stuff out?

Ben Yelin: I think so. I mean there are a couple of ways you could potentially try to ameliorate this problem. One of them is having more human involvement in the process. So, you hire the agent, the AI agent produces your travel itinerary. It sends it over to the human and says, "Do you want to check this? Do you want to verify it?" But that kind of defeats the purpose of having an AI agent, doesn't it? Like, if I'm going to be involved as a human, why shouldn't I just just make all of the arrangements myself? It's not actually saving me that much time If I have to double check the work of the AI agent.

Dave Bittner: Yeah.

Ben Yelin: So, yeah, I do think, like, if we're actually going to use these agents as their designed, which is to limit human involvement to the greatest extent possible so that we can focus on whatever other brilliant things we're doing with our lives, then yeah, I do think this is -- I do think we're kind of on a collision course here. I think we'll hit a breaking point where there are enough hallucinations or errors that lead to significant damages, that lead to lawsuits, and courts are going to start to hold these developers liable and that will have a big impact on the market one way or another.

Dave Bittner: And the developers --

Ben Yelin: I don't see how we don't get there unless the developers get so good that they minimize the errors to such a significant extent that you have a 99.999% success rate and there aren't the types of grounds for lawsuits that are going to be successful. But we're certainly not there yet. At least according to the evidence we've seen in this article and others.

Dave Bittner: Yeah. And it could just be worth it for the -- it will either be worth it or it won't, right? In other words, it will be worth it for the companies, the companies will get so good at it that the success rate is high enough that even if they have to pay out the times when it screws up, there's still so overwhelmingly profitable that it's worth it. Or it won't be.

Ben Yelin: Right, exactly.

Dave Bittner: And they'll put guardrails on it that say, you know, you'll ask it to plan your vacation itinerary and it will come back and say, "Ah, no, no, no."

Ben Yelin: I've gotten trouble for doing this in the past. You are not going to get me to do this.

Dave Bittner: Right.

Ben Yelin: Yeah, that's a good point. Maybe at some point it is so profitable and becomes to ubiquitous that they're willing to accept whatever legal hits they get from people. If you have a 99.999% whatever, success rate, maybe you just settle with the people who get hurt by it. Give them a bunch of money and trust that the rest of the system is working properly so that you can keep raking in the money. Maybe that's the other equilibrium here.

Dave Bittner: But it's interesting to me that in this case the conversation you and I are having are that it's pretty clear that the company who runs the AI is responsible for the mistakes.

Ben Yelin: I think so. I think --

Dave Bittner: Different from the suicide.

Ben Yelin: Yes. I think that owes to the specific circumstances here. II think agents are giving clearer instructions. It's, again, I don't even know how to really explain this or conceptualize it. But, like, there's a closer relationship between a human's decision making and the output of the AI. Whereas it took months for that kid to develop a relationship with a chatbot on LLM and the kid had as much of an influence on that chatbot as the original developer because the chatbot is reacting to that kid's personality and what that kid is telling it. So, it's continually training itself through that relationship. I don't think that's quite analogous to what's happening here. I think these are -- this is more like traditional computing where you're giving it complicated instruction that involve human-type processing but it's really kind of like if X, then X. If not X, then Y. Like, multiply that by a million different decision points. Does that make sense at all?

Dave Bittner: Yes. I do see -- yes, I think there is a -- there's definitely a distinction.

Ben Yelin: Where to draw that line, I don't know. Like, is it -- how attenuated is attenuated. Like, where is the exact point where you can't hold the developer liable because the chatbot has evolved into something that the developer couldn't have possibly foreseen.

Dave Bittner: Right.

Ben Yelin: Is it a time period? Is it a number of conversations? I have no idea. I wouldn't even venture to guess.

Dave Bittner: Let me just got to a dark place briefly. Suppose one kid bullies another kid into committing suicide. What's the legal peril of the bully kid and the bully kid's parents?

Ben Yelin: I mean you can -- there's potentially a lawsuit there if it's directly traceable.

Dave Bittner: Is there a crime?

Ben Yelin: It is really circumstanced dependent. It most contexts I don't think it's -- like, bullying itself in the absence of physical violence is probably not going to lead to criminal liability on the part of the person or the parents.

Dave Bittner: Okay.

Ben Yelin: If it's directly traceable, like you can prove that it was the bully's actions or the bully's words that led to the suicide, then I think you could have a lawsuit for wrongful death or intentional infliction of emotional distress. But that's going to be a really hard case to bring. But it's a good metaphor. I mean I could see why it would be applicable in this case where maybe it's a factor, maybe it's the overwhelming factor but there are still cases where you're not going to hold that bully, or that bully's parents liable depending on what type of evidence you have that it was the bully's actions that weren't constitutionally protected free speech that led to what happened with the kid.

Dave Bittner: Well, that's where -- yeah, does the bully have free speech protections, where as the AI may not?

Ben Yelin: Yes. Now, not to get into the -- this is getting really in the weeks but like public school students have a decreased level of First Amendment protection relative to adults --

Dave Bittner: Right.

Ben Yelin: Just because public schools have an interest in maintaining order, etcetera, that sort of thing.

Dave Bittner: Yeah.

Ben Yelin: So, it's not the equivalent free speech rights that any adult would have in being protected in what they say.

Dave Bittner: Sure.

Ben Yelin: And then certain types of speech doesn't fall under First Amendment protection at all. So, true fighting words that lead to direct violence and altercation, any words that would immediately cause imminent lawless action, like that's not constitutionally protected speech. But a bully just saying mean things to a kid, that's going to be almost an impossible thing to prove.

Dave Bittner: Right, subtle psychological manipulation, which I think in the realm of what they're accusing the AI of having done.

Ben Yelin: Yeah, I mean if you look at the exact words that the AI said, it didn't explicitly say you should commit suicide now. I mean it encouraged it in so many words. Come home to me as soon as possible. But this is when the 14 year old had prompted the AI saying that this is something he had been thinking about.

Dave Bittner: I see.

Ben Yelin: So, I don't know. That is also a very difficult parallel.

Dave Bittner: Yeah, absolutely. How do you suppose -- well, let me phrase it this way, what's our trajectory here. Like, is this are we one ten year path, are we on a 20 year path, are we on a one year path for any of this being clarified?

Ben Yelin: I think we're on a shorter timeline in terms of actions that the companies are going to take themselves to try and avoid any legal liability.

Dave Bittner: So, they foresee the potential here.

Ben Yelin: Totally.

Dave Bittner: They're going to try to prevent it. Yeah, that makes sense.

Ben Yelin: And even to give them credit, like I think they understand the moral dimension here, like, there are only so many of these types of stories that can be out there without there being a significant public outrate and increased demands on these companies to put up greater guardrails.

Dave Bittner: Yeah, that's fair.

Ben Yelin: And in this article, we've already seen that both of the companies that are named in the lawsuit relating to the 14 year old have already taken actions to try and come up with a child safe version of an LLM that has greater guardrails. And I think that's kind of the best thing for everyone. We don't have to deal with all this litigation but companies have been scared straight into putting up proper guardrails. My concern is that in other circumstances that maybe are a little bit more unclear, maybe even circumstances when children aren't involved, it would be so much more expensive and burdensome for the companies to make changes or put up guardrails for such a diffuse chance that there's going to be some legal harm that somebody is going to take violent action against themselves or others as a result of what that chatbot has said that they just might not bother with raining themselves in. And that's where I thing governments have a role in forcing these companies to put on guardrails to prevent these types of things from happening. And I think we'll start to see more, specifically state legislation where it imposes liability or there might be an affirmative defense in cases like this if the developers of these chatbots put in place certain safeguards. We've seen that in a cybersecurity context where companies have an affirmative defense against data breaches if they've taken certain cyber hygiene measures.

Dave Bittner: Right.

Ben Yelin: And so maybe we could see something similar on these types of cases.

Dave Bittner: Yeah. All right, well that's an interesting line of discussion, isn't it?

Ben Yelin: Can I also just say, at the end of this article, it will post in the show notes on the Washington Post --

Dave Bittner: Yeah.

Ben Yelin: They put in, which I think should be put in any article about this, that if you or someone you know needs help, visit 988lifeline.org or call suicide and crisis lifeline at 9-8-8. I think it's an important reminder.

Dave Bittner: Yeah, absolutely. All right, we will have a link to the stories in the show notes. And of course, we would love to hear from you if you have some feedback or something you'd like us to consider for the show, you can email us. IT's caveat@N2K.com. [ Music ] And that is "Caveat" brought to you by N2K CyberWire. This episode is produced by Liz Stokes. Our Executive Producer is Jennifer Eiben. The show is mixed by Trey Hester. Peter Kilpey is our publisher. I'm Dave Bittner.

Ben Yelin: And I'm Ben Yelin.

Dave Bittner: Thanks for listening.