AI on the witness stand.

[ Music ]

Dave Bittner: Hello, everyone, and welcome to "Caveat," N2K CyberWire's privacy surveillance law and policy podcast. I am Dave Bittner and joining me is my co-host Ben Yelin from the University of Maryland Center for Cyber Health and Hazard Strategies. Hey there, Ben.

 

Ben Yelin: Hello, Dave.

 

Dave Bittner: On today's show, Ben has the story of law enforcement agencies increasingly relying on AI to synthesize digital evidence. I've got the story of the Secretary of Commerce pressuring Taiwan over chip manufacturing. And, later in the show, my conversation with Sarah Graham from the Atlantic Council's Cyber Statecraft Initiative discussing their work and findings on "Mythical Beasts: Diving into the depths of the global spyware market." While this show covers legal topics and Ben is a lawyer, the views expressed do not constitute legal advice. For official legal advice on any of the topics we cover, please contact your attorney. [ Music ] All right, Ben, we've got some follow-up this past week.

 

Ben Yelin: It's some good follow-up.

 

Dave Bittner: It is.

 

Ben Yelin: Yeah.

 

Dave Bittner: It's from a listener named Kevin who writes in and says, "I got so angry when Ben said that if it doesn't have the consent of all 100, you'd have to jump through a bunch of procedural hoops which Senate leaders just don't want to do. It's very time consuming." Kevin says, "This confirms what I have said about our House and Senate, politicians are always putting their energy into winning elections or helping others. Ben, fix this or tell us how." Ben?

 

Ben Yelin: I don't have a good answer for you, Kevin, 'cuz I think you're absolutely correct. I mean, they enjoy fundraising. A lot of their job is fundraising.

 

Dave Bittner: Yeah.

 

Ben Yelin: They'll be on the phone all day talking to potential fundraisers, especially if they're running for reelection. You'd think in the Senate that would be a little bit less acute 'cuz only a third of the senators are running for reelection in the next two years. But, these days, you know, election cycles are six years. You're already thinking about what the political climate's going to look like in 2030.

 

Dave Bittner: Right.

 

Ben Yelin: So a lot of that is fundraising. A lot of it is they like to be home with their constituents. One of the -

 

Dave Bittner: Do they, though, Ben? Do they?

 

Ben Yelin: I mean, I guess it depends on how you define constituents.

 

Dave Bittner: Okay.

 

Ben Yelin: They like to be visible in their states and districts -

 

Dave Bittner: Right.

 

Ben Yelin: - and meeting with people who they think can help them achieve their priorities, -

 

Dave Bittner: Yeah.

 

Ben Yelin: - whatever those are.

 

Dave Bittner: That's fair.

 

Ben Yelin: But one kind of axiom about Congress is the one thing that can make them move is the smell of jet fumes. If they think like, "All right, if we just come up with a deal here, I can leave," that's the one thing that has seemed in the past to inspire them to come up with compromises.

 

Dave Bittner: Ah, I see, the big motivator is that they might have to stay longer than they had planned in D.C.?

 

Ben Yelin: Yeah. It's - it is really interesting. It's not like this in state legislatures. Just having followed the Maryland General Assembly, all the members are present for committee hearings for the most part, they're present for floor sessions. They will work through legislation until midnight on some nights. And, honestly, legislatures in states function better than the United States Congress so maybe they're doing something right.

 

Dave Bittner: All right. Well, thanks to Kevin for sending in that comment. We do appreciate it. And, of course, we'd love to hear from you. If there's something you'd like us to consider for the show, you can e-mail us. It's caveat@n2k.com. All right, Ben, let's jump into our stories here. You want to kick things off for us?

 

Ben Yelin: Sure. So there is a revolution happening in - within law enforcement agencies across the country and that is the use of AI to sift through digital evidence. So there are a number of startups who have provided software to AI companies. There's this Washington Post article we can put in our show notes about Redmond, Washington, which has purchased a software entitled Longeye, a San Francisco startup, go SF, which offers an AI chatbot designed to answer questions about what is inside tranches of digital evidence to help uncover missed connections and incriminating statements. So I guess we'll start with what happens in a non-AI world is there is a giant stack of papers or, if you're lucky, you can utilize, you know, Control-F in a giant digital document.

 

Dave Bittner: Right.

 

Ben Yelin: You're looking for pieces of relevant evidence to your case, so did this person make a phone call to a criminal suspect, where were they at this time according to these cell site location information records, what did they say in a recorded conversation, what did they say in a deposition. All of those go into a vast tranche of case files and it can be very, very time consuming to read through all of those case files. Sometimes there are hundreds and hundreds of pages. Incidentally, this is the work that's done by a lot of entry-level legal professionals. So, if you can't find a job coming out of law school, one thing you'll almost certainly be able to do is temporary document review work where a law firm will hire you, they have a big case, you look through hundreds, if not thousands of pages of evidence and try and highlight whatever it is that they are looking for.

 

Dave Bittner: Sounds fulfilling.

 

Ben Yelin: Extremely fulfilling. But you know what? At least it's a job, right?

 

Dave Bittner: There you go. Paying your bills is very fulfilling.

 

Ben Yelin: Yeah. I had a temp job out of college and -

 

Dave Bittner: Yeah.

 

Ben Yelin: - it was not the most exciting thing in the world, but I could pay the bills.

 

Dave Bittner: Sure.

 

Ben Yelin: And, you know, there's something important about that, right?

 

Dave Bittner: Yeah, absolutely.

 

Ben Yelin: The change here is that AI is gonna start doing a lot of the work. You can instruct the chatbot to say, "Look through these documents and find X," so "Find any evidence that this phone call - this phone number called that phone number" or "find anything within these transcripts that might be incriminating in this way." And that will significantly cut down on the amount of time that police departments have to devote to sifting through digital evidence. And that's gonna have a couple of really big impacts. The one I talked about, which is probably of lesser importance, is it might not be good for the job market for recent law school grads. I realize -

 

Dave Bittner: Right.

 

Ben Yelin: - that's a niche concern, but certainly something worth noting. The second I think more bigger implication is this gives law enforcement powers that they did not have in the past. They were constrained by just the process of sifting through digital evidence. There's only so many resources you can employ and not every case you're gonna be able to actually sift through all thousands, if not hundreds of thousands of papers of evidence. Police departments have a lot of cases. It's not like "Law and Order" where for 40 minutes each week, they devote the full force of their entire agency to solve a single crime.

 

Dave Bittner: Right.

 

Ben Yelin: It doesn't work like that, especially in higher crime jurisdictions. So, prior to AI, they might have had to use man hours that were just completely inconceivable to review these types of documents. And the same is true for defense attorneys for that matter. That might no longer be the case and that, to me, has major Fourth Amendment implications. There's a theory of Fourth Amendment jurisprudence put forward by our man, Mr. - Professor Orin Kerr. Still invited on this show if you're listening.

 

Dave Bittner: Right.

 

Ben Yelin: One day - one day we're gonna do it. And it's called the equilibrium-adjustment theory. Basically, as technology improves for either criminals or law enforcement, the law should or does adapt to restore the same level of Fourth Amendment protection as existed prior to the advent of that technology. So judges are now gonna be aware that it's much easier to search through digital evidence than it once was. And that might cause them either explicitly or implicitly to come down on the side of raising standards for law enforcement agencies for them to obtain a warrant or for them to carry out some type of operation. I think if they see that this will be such an advantage for police departments that in order to maintain the same level of protection - Fourth Amendment protection for criminal suspects, they're gonna have to tilt the scales towards the criminal suspects in some way through changing jurisprudence. And we've seen this in a number of different cases, Riley versus California in 2014 when they confronted the case of whether you need a warrant to look through somebody's cell phone upon arrest. Prior to that case, the thinking was, "Of course, if you're arresting someone, for your own safety, you should be able to look at everything that they have on them, make sure that they're not carrying contraband, make sure they're not carrying weapons." But cell phones are different. Cell phones reveal the most intimate aspects of our lives. They contain all of the information that you can possibly imagine without us. So the Supreme Court said, "Because the technology has changed, our jurisprudence has to change and now a warrant is required to search the contents of a cell phone." So I think we could see something similar happen in response to the proliferation of AI chatbots in law enforcement agencies.

 

Dave Bittner: So I'm reminded of that old, I don't know, I guess it's a chestnut, I have heard law enforcement officers say, "If I want to pull you over, all I need to do is follow you long enough and you'll do something."

 

Ben Yelin: Yeah.

 

Dave Bittner: That -

 

Ben Yelin: Unless you're a pristine driver, which most of us are not.

 

Dave Bittner: Right. But - and so they just, you know, bide their time. With enough patience, they'll come up with some kind of justification for pulling you over. This reminds me of that in a certain way where what if I have the contents of someone's cell phone and I say to my AI, "Find something illegal in here."

 

Ben Yelin: Yeah. I mean, that's - I think that's a great example because you can't have law enforcement agents for 24 hours a day surveil individual criminal suspects. It's just not practical. I guess they could do one, they could do several, they could do a handful of criminal suspects, but there just are limited resources if they wanted to do broad-level surveillance of that kind. But, here, with AI, you could be skipping those steps.

 

Dave Bittner: Right.

 

Ben Yelin: Now, we're talking about digital evidence so that's different than following somebody in their car. But, as you said, they could just put in a prompt that says, "Look through these transcripts, look through these documents, look through this person's actions or whatever and find evidence of a crime." And that process is gonna allow you to uncover more criminal activity in a shorter period of time than in a pre-AI world where you would have had to do the legwork of actually reading through hundreds, thousands of pages to find that evidence. Maybe it doesn't make a difference in the most high-profile cases where people were gonna be combing through that evidence anyway. But, when you apply this at scale, it's just gonna probably lead to there being more criminal prosecutions because you're gonna be able to find more incriminating evidence than you otherwise would have been able to find.

 

Dave Bittner: Isn't this the phishing expeditions that our founders feared?

 

Ben Yelin: Yes, it is. I mean, this sounds similar to general warrants. And I think it's possible that if we see some case come up in federal court about this that they would make that analogy saying that maybe you have to have some type of reasonable suspicion or probable cause to conduct certain AI queries. We're a long way from that happening, but this is sort of a preview of where I see the future going where you wouldn't be able to query without any suspicion, "Find me evidence that Dave Bittner committed a crime based on this voluminous document." You might have to say, "I'm looking for evidence - I have probable cause to believe that Dave committed espionage."

 

Dave Bittner: Right.

 

Ben Yelin: "And I'm - please find evidence that would indicate as such."

 

Dave Bittner: I see.

 

Ben Yelin: We're a long way from that, but that's kind of what law enforcement has tried to enforce in other contexts, a certain type of particularity. And I think we could see that here.

 

Dave Bittner: What do you think it's gonna take to get judges up to speed on this? 'Cuz, as I can imagine, the defense lawyers throwing all kinds of shade at this, "There's hallucinations and, you know, how - we can't trust the machines, Your Honor."

 

Ben Yelin: Yeah, I completely agree with that. I think it's gonna be a major gift to defense attorneys because it could sow reasonable doubt. AI does hallucinate. There have been high profile cases of hallucinations in criminal proceedings. So this is something - at least the judges that aren't like 95 years old, this is something that judges are aware of.

 

Dave Bittner: Right.

 

Ben Yelin: And so I think that would be pretty compelling. I think it's incumbent upon these law enforcement agencies and upon the manufacturers of the software to make it clear that this is not a 100% foolproof system and that there has to be some level of human oversight. So the company that's mentioned in some of these articles, they have a feature where, when it spits out the evidence that you're looking for as a result of that chat, it will cite to the place in the document where that evidence exists. So it allows the human to go in and look at that evidence.

 

Dave Bittner: Right. And there's the job for your college grad.

 

Ben Yelin: Yeah. But you probably don't need as many of those people -

 

Dave Bittner: Right, right.

 

Ben Yelin: - if AI is doing so much of the work for you.

 

Dave Bittner: But you can see the acceleration that this could provide.

 

Ben Yelin: Yeah, it could totally accelerate the ability to search these things. And then, if you actually - if it's actually directing you to the evidence itself, then the work of the lawyer is very different than it previously has been. It's -

 

Dave Bittner: Right.

 

Ben Yelin: - let's analyze the raw data here and see if that really is incriminating evidence based on, you know, the three years I spent in law school and my however many years of experience.

 

Dave Bittner: Yeah.

 

Ben Yelin: And that's, frankly, more valuable analytical work than "read through these thousands of pages and see what you can find." It's probably not - as I said, this is something that may - might be good for incumbent lawyers who are gonna have more interesting work. And that's true for detectives who work in law enforcement agencies, they're gonna be able to spend more time solving crimes and less time sifting through digital evidence. And that's good for people who have been there a while. But, for entry-level positions, you know, I think that's one place where law enforcement or law firms can start to cut hiring, which is one thing I would be fearful of as a result of this.

 

Dave Bittner: It's interesting to me, I think of this as, on its face, you think, "Well, this is great. You know, we'll be able to solve more crimes more quickly. Police officers will be able to spend more time making their communities safer, rather than the drudgery of, you know, going through all the technicalities and all those sorts of things." And yet I can also envision the unintended consequences, the societal implications that we've already seen from technology, right, that changes the way law enforcement functions and we're not always okay with that.

 

Ben Yelin: Yeah. I mean, I think one of the potential implications, and this is a shot in the dark 'cuz I don't know exactly what's gonna happen, is there are gonna be more crimes that would have never been discovered that are now gonna be prosecuted. I think about something like perjury. Somebody makes a statement at a proceeding where they're under oath and law enforcement or whomever the investigator is wants to prove that this person committed perjury. This is something that would have required a lot of work. I mean, you have to go through a voluminous record to see whether somebody said something that contradicted the statement they made under oath or if there's evidence anywhere that contradicts the statement the person made under oath. And that is gonna be a lot easier.

 

Dave Bittner: Right, right. Say, "Hey, go through every publicly available transcript of anything this person has ever said and see if any of it contradicts this statement."

 

Ben Yelin: Right. And some of them can be not innocuous statements, but statements that aren't in and of themselves that incriminating of serious crimes, but you could still go after somebody for perjury. And, oftentimes, that's a gateway into getting them to cop to other crimes.

 

Dave Bittner: Yeah.

 

Ben Yelin: Or it's a way to get suspects to cooperate against their co-conspirators.

 

Dave Bittner: Yeah.

 

Ben Yelin: We say, "We have you on perjury so now you need to give us the goods on your co-conspirator." And that will make law enforcement work a little bit easier. So, yeah, I mean, we might uncover evidence of crimes that just would have never been prosecuted in the past. There's still gonna be a level of prosecutorial discretion 'cuz, when you're actually making the case, that's pretty labor intensive. Going to court is labor intensive. But maybe people who previously you would have taken to trial, the evidence is gonna be so overwhelming against them that they might take a plea deal. So this is just what I could foresee happening.

 

Dave Bittner: Yeah.

 

Ben Yelin: Who knows. We're still at the very early stages of this. Not that many law enforcement agencies are currently making use of this type of technology. But I do think it's like a pretty good use case of AI, especially if it's confined to a closed universe. Like it's not asking you to scrape the internet to find evidence of a - that somebody committed a crime. You're putting otherwise - or you're putting digital records in whatever AI service you are using and they are combing through those records.

 

Dave Bittner: Right.

 

Ben Yelin: So that decreases the chance that you'll see things like hallucinations.

 

Dave Bittner: Yeah, all right.

 

Ben Yelin: The world is changing, my friend.

 

Dave Bittner: It really is, it really is. All right, I'll tell you what, let's take a quick break to hear from our show sponsor. We'll be right back. [ Music ] And we are back. My story this week comes from the folks over at Ars Technica. I have to say, it strikes me as kind of weird. That's the - that's my highly technical and -

 

Ben Yelin: Sophisticated intro there?

 

Dave Bittner: Yes.

 

Ben Yelin: Yeah.

 

Dave Bittner: Exactly. So the article is titled "Taiwan pressured to move 50% of chip production to US or lose protection." So the backstory here is that the vast majority of the production of high-tech chips happens in Taiwan. The chips that are in your Apple devices, for example, chances are that those are manufactured in Taiwan. It's one of the things that Taiwan does best and arguably does better than anyone else in the world. This is one of the reasons why Intel is struggling here in the U.S. is that they've been unable to keep up with the advancements that the Taiwan companies have had in chip production.

 

Ben Yelin: Right. Which is why we had to purchase a portion of Intel, correct?

 

Dave Bittner: Yeah, well, we - well, I will say we purchased a portion of Intel. Did we really - did we purchase it or was it in exchange for grants or loans? It's all very messy, Ben.

 

Ben Yelin: Well, we won't speculate.

 

Dave Bittner: It's all - we won't speculate. It's all very messy. Let's just say the bottom line is the federal government now has a stake in Intel. And so the Secretary of Commerce, Howard Lutnick, has told his Taiwanese counterparts that he would like to see 50% of chip production move to the U.S. or else. What's the "or else"?

 

Ben Yelin: Quid pro quo here.

 

Dave Bittner: Right. What's the "or else"? The "or else" is that they could lose the protection of the United States in the event of Chinese aggression.

 

Ben Yelin: I mean, these are pretty heavy-handed tactics because presidents of both parties and members of Congress have spent decades being concerned about Taiwan and a potential Chinese attack or invasion.

 

Dave Bittner: Right.

 

Ben Yelin: We are allies of Taiwan, obviously. There are a lot of politicians, Nancy Pelosi is one of them, who one of the key causes of their career has been the plight of the people of Taiwan. And now we might sacrifice all of that to move 50% of chip production to the United States. It's a very interesting threat to say the least. I mean, these are tactics that generally you would see in organized crime.

 

Dave Bittner: That's why I say it's weird. It just - I mean, am I off base here that elements of this are weird?

 

Ben Yelin: I think a signature of Trump policy, which this conforms to, is trying to use leverage in international affairs that we've never tried to use in the past because of customs or because of the potential fear of blowback or just because it was morally wrong. And I think now everything - everything we do in international affairs feels more transactional like we are trying to extract a price to get something that we want and we're willing to blow stuff up if the other country doesn't comply with us.

 

Dave Bittner: Yeah.

 

Ben Yelin: A lot of people really like this. I mean, this is - this is a - this is an American First foreign policy.

 

Dave Bittner: Right.

 

Ben Yelin: You are threatening to undermine the security of Taiwan, something that we've sought to guarantee for a long time in this country. Even President Trump was known as a major China hawk in his first term. Right?

 

Dave Bittner: Yeah.

 

Ben Yelin: And one of the reasons, at least ostensibly, he was not interested in supporting Ukraine is that that might be a distraction or it might divert resources from being adversarial towards China. And here we are. So, yeah, I just think that this is their way of doing things. It comports with what a lot of his supporters love about him which is that he's willing to play hardball to protect American economic interests. I think that's true. Whatever you think about this from a moral perspective, I'm not - obviously not a huge fan of this type of blackmail, but I think this does conform to a very consistent philosophy that we've seen this second term.

 

Dave Bittner: It also strikes me just from a negotiation point of view as being unrealistic because my understanding is that it's not like you can just build a factory over here and start manufacturing these things. The amount of expertise, the supply chains, all of the things that go into -

 

Ben Yelin: The raw materials, right?

 

Dave Bittner: Right.

 

Ben Yelin: Yeah.

 

Dave Bittner: Right. And, if it were easy, Intel would already be doing it and it's not. And so this idea that they can just pick up and shift their production over here to the U.S. I think many people would say is folly. And this article describes it as Herculean that it's not easy. So, again, back to the negotiating tactic, we're asking Taiwan to do something that it probably can't do.

 

Ben Yelin: Yeah. It's also like - and this is gonna make me sound like the proverbial neoliberal shill. But the current arrangement is kind of good for everyone. Like it's cheaper to produce these in Taiwan, -

 

Dave Bittner: Right.

 

Ben Yelin: - which means that we can get them cheaper. This is an industry that is powering the Taiwanese economy, which is obviously good for them.

 

Dave Bittner: Yeah.

 

Ben Yelin: When we've tried to onshore certain things, generally, they have become more expensive. Now, there are trade-offs that are involved there. Offshoring has led to the decline of a lot of different jobs, especially in the industrial Midwest, for example. And that's devastated communities. So this isn't an easy call by any stretch of the imagination. The only way it would make sense for us to ruin this financially beneficial relationship is if there really were a national security concern about chips being made overseas. I think there are national security concerns about things that are manufactured in China, which is our political adversary. That was kind of the whole shebang behind this TikTok kerfuffle -

 

Dave Bittner: Right.

 

Ben Yelin: - is that ByteDance was basically an agent of the adversarial Chinese government so we had to tread carefully. But Taiwan is an ally. They are a democratic - small d, democratic ally. It's an ally that we've pledged to support. So, in that sense, it just - it doesn't make that much sense to me.

 

Dave Bittner: Suppose we pull back our support for Taiwan, what are the possibilities of them looking towards, say, the European Union for protection?

 

Ben Yelin: I think the possibilities are quite high. They will turn not just to the European Union, but, for other emerging countries, perhaps even some in Southeast Asia, I think there are markets out there. Obviously, they take a big hit, the U.S. is a huge market, they're a huge buyer, but there are alternatives out there. And, if it turns out that Taiwan is insulted by this type of quid pro quo, that they don't comply with the demands of this negotiation, but they do decide that they want to have a more adversarial relationship with the United States, then it's kind of a lose-lose. Like they might decide, "Okay, we'll replace your market share with European Union countries. They are gonna guarantee our security and you are not gonna onshore production of chips in the United States anyway."

 

Dave Bittner: Right.

 

Ben Yelin: So that's kind of the worst-case scenario of them calling our bluff. I think the Trump administration is convinced that they are reliant enough on not just our security protections, that those can't be replicated by European Union countries, and the same thing is true for the market for these types of chips.

 

Dave Bittner: I'm just imagining, you know, Tim Cook at Apple looking at all this and thinking, "Oof."

 

Ben Yelin: Yeah, he's keeping his mouth shut though. I mean, -

 

Dave Bittner: Yeah, we don't wanna go - yeah, he's thinking, "We don't wanna go back to Intel." But, yeah, interesting times.

 

Ben Yelin: I mean, he has some very expensive jewelry that he'd like to give to the White House to influence policy, so.

 

Dave Bittner: That's right.

 

Ben Yelin: Sorry, Tim Cook. If you're listening, I still love you.

 

Dave Bittner: A solid gold Apple Watch.

 

Ben Yelin: Yep. [ Laughter ]

 

Dave Bittner: All right, we will have a link to that story in the show notes. We'll be right back after this message from our sponsor. [ Music ] Ben, I recently had the pleasure of speaking with Sarah Graham from the Atlantic Council's Cyber Statecraft Initiative and we are discussing their work on a publication they recently released, it's called "Mythical Beasts: Diving into the depths of the global spyware market." Here's my conversation with Sarah Graham. [ Music ]

 

Sarah Graham: So spyware can be defined in a few different ways, but, in our report and this second published report we have here, we take a relatively narrow scope to how we're defining spyware as a type of malicious software that enables the unauthorized remote access of a target device for the purposes of intrusion and surveillance. And so, oftentimes, folks will ask us, "Why isn't the scope broader" or "did you include something like stalkerware or adtech?" And we really keep a relatively narrow definition here focused on that unauthorized remote access specifically for the purposes of surveillance.

 

Dave Bittner: Well, can you paint a picture of this market for us here? I mean, who are the key players and how do they tend to operate?

 

Sarah Graham: Sure. So, in terms of the marketplace, in recent years, there's been a lot of great reporting by different organizations, including Citizen Lab and Amnesty Tech, about the harms coming out of the spyware industry. But, in contrast, there's been a relative lack of transparency about what's happening within the marketplace. We oftentimes see headlines about big players, such as the NSO group, but there's a lot of other players small, medium size and more than just vendors. So our report is really trying to turn to look at the marketplace as a whole and understand the supply chain of different vendors, holding companies, investors and the like across the marketplace at a global scale.

 

Dave Bittner: Well, who are some of the key players here?

 

Sarah Graham: Yeah. So, in our report, we really take a look at the global scale of the spyware market, but hone in on a few particular areas of interest. On first turn, one of our major trends that we identified in the original report and see holding consistent with data from this past year are that the majority of identified entities are domiciled in Israel, India and Italy. But something I really want to point out and we see as a trend emerging in this updated dataset is that there has been a significant increase within our sample of U.S.-based investors that continue to disproportionately fund capabilities. And, really, we want to highlight this because it really undermines the important U.S. government action that we've seen in the past 18 months to two years on spyware in contrast with the investors from the U.S. who continue to invest in these types of technologies.

 

Dave Bittner: Yeah, can we dig into that a little bit? What - what insights can you provide as to what's driving that market investment from the U.S.?

 

Sarah Graham: So, to start with a bit of data, we see with the data from this past year a total of 31 total U.S. investors that we were able to identify. And this - we find this is particularly remarkable given that some of the entities that their - that U.S. investors are investing in are actually listed on the U.S. entity list. So, for example, Candiru listed on the entity list has investment from U.S. firms including Integrity Partners, as well as a few other venture capital firms and pension funds coming out of different states. So we're seeing that there is investment from a wide range of investors into a wide range of different spyware vendors, not only including Candiru, but Cognant and a few others as well. And, to your question of what's really driving this, we don't - our data doesn't necessarily answer that question, but something that my fantastic co-author Jen Roberts continues to point out as we have dug into this data is there has to be a reason. Investors invest when they see the potential for profit, despite U.S. policy actions, such as the entity list. There obviously must be some expectation that there is a future profit to be had here.

 

Dave Bittner: Yeah. Well, help us understand how these commercial spyware vendors differ from traditional state intelligence operations.

 

Sarah Graham: One thing that we get asked about quite a bit is, "How do these spywares differ and how are they useful to governments for national security?" And there's a strong sense that there is some political will to put controls on these technologies despite the fact that they are used for permissioned uses in state national security operations. But there is interest to preserve that limited use. And the ability to do so really hinges on these sorts of transparency efforts with datasets like the ones that we've created here to ensure and maintain the integrity of those capabilities for those narrowly defined and permissioned use cases.

 

Dave Bittner: Well, one of the things you uncover in the report here digs into the government's role, both as buyers and regulators of this.

 

Sarah Graham: Sure. So the government - the state plays a different role in different jurisdictions here. And so I can give you a few examples. As I mentioned at the start, what we do observe and see consistent is that three jurisdictions, in particular India, Italy and Israel, have a lot of activity within their boundaries. And what is something that we see is consistent amongst these three is that it's a relatively permissive environment with some sort of state involvement. This varies. We see in the Indian cluster that this is most common in the sort of hack for hire market. In Italy, there's a much older history of spyware with quite a bit of overlap with state entities, whether that's as buyers, - well - but also as regulators, which, as you can imagine, can oftentimes create some either implicit or explicit conflicts of interest and can really make transparency efforts and ultimately any meaningful regulation quite challenging.

 

Dave Bittner: Why is it so difficult to hold these spyware companies accountable? I mean, is it as simple as the fact that they're offshore from our own local regulators?

 

Sarah Graham: That's the million-dollar question. I think a lot of people would like to know the answer to this. One - I can point out to a few things.

 

Dave Bittner: Yeah.

 

Sarah Graham: One trend that we see in all of our reporting has been this feature of shifting vendor identities. So that might be really subtle name changes or total rebrands of different entities that make it really difficult to track consistency in their activity. These entities have a lot of smart folks involved and so strategic jurisdictional hopping is something that they certainly partake in. And we see this in a few different examples. For example, we know from a court case that QuaDream established a presence in Cyprus to avoid European export controls. And so, as you can imagine, this sort of limited ability to consistently track is a huge barrier. And that actually highlights one of our second key trends in this report around the role of resellers and brokers, which I'd be happy to talk about a bit more, too.

 

Dave Bittner: Well, yeah, let's dig into it. What can you share about that?

 

Sarah Graham: Yeah. What we find with our updated data sample is a large number of what we're calling resellers and brokers. These are sort of partners within the marketplace that can be unrelated to the development of spyware, but they contribute some sort of technical or business need for the vendor. So this could be something like marketing services, the provision of telecommunications intercept devices or creating access to some sort of regional market that an original vendor might not have otherwise been able to easily enter and, you know, sell to interested buyers. And so through access to public data sources, we've been able to identify a larger share of these entities and have come to identify more and more that, in this expanded and opaque marketplace, these types of intermediary entities are playing a pretty crucial role to limiting transparency.

 

Dave Bittner: Yeah, can we talk about that? I mean, one of the things that it strikes me is that these spyware companies give a lot of nation states plausible deniability, right? I mean, to what degree is that an element here?

 

Sarah Graham: Certainly. There is plausible deniability sort of up and down the supply chain. You could imagine all the way through at the end use of surveillance plausible deniability in terms of how particular information is gained. But there's also plausible deniability upwards on the supply chain. And I think these brokers and resellers play a really crucial role to that, whether it's mere overlap only of business officers or whether that's some sort of larger overlap between, for example, the original vendor and setting up some sort of satellite office in another state to gain access to that market. Some examples that we really dug into in the report are around the Mexican spyware ecosystem. And, with some recent transparency reporting, we've been able to see how there were layers and layers of plausible deniability through the creation of contracts that only very subtly indicate, perhaps, that these technologies were being sold to different state agencies.

 

Dave Bittner: How hard was it to put together this report? You know, you and your co-author. What are some of the challenges that you faced in gathering and assembling this information?

 

Sarah Graham: There's quite a few. Our report relies on publicly accessible data, which we extensively link throughout the interactives that anyone can go online and see. One of our main data sources are corporate registries and oftentimes these are state-hosted corporate registries. Some states have really fantastic registries. They're well-documented, they're not paywalled. A great example here is the UK corporate registries. We actually found a lot of valuable information in some Ecuadorian ones as well. Whereas, on the other hand, there are some states where these registries either are non-existent, don't actually register information through search queries or for another - you know, a variety of reasons are not particularly useful for our data collection measures. So, obviously, that sometimes spews our dataset because we have more information on entities and jurisdictions where such data is more accessible.

 

Dave Bittner: Yeah, that's fascinating. What are some of the trends that really have you concerned about the coming years as we look toward the horizon and, you know, the long-term implications of spyware?

 

Sarah Graham: I think one trend that at present and looking ahead, I am not sure - and I've spoken with my co-author Jen about this quite a bit, I'm not sure we see much indication of change is around this U.S.-based investment. So, despite all of this policy action, including visa restrictions, sanctions, executive orders coming out of the U.S., there's still clearly a disconnect between the American investment community and the American policy community on this issue. And so, without any sort of meaningful movement on the needle there, I think that this is something that could continue to persist or - and potentially expand in scope.

 

Dave Bittner: Where do we stand with policymakers here in the U.S.? Is there any broad agreement on the place that spyware is intended to play?

 

Sarah Graham: With the current - with the new administration, it's actually, you know, still relatively early within the tenure of four years and we haven't seen any public indications of change to the current status quo of the U.S. policy perspective on these issues. But I think something to point out is that the absence of any change suggests at least that, at minimum, the current trajectory, which has in the past included this effort through listing vendors on the entity list, issuing sanctions and visa restrictions, just given that these things have not necessarily been pulled back is some sort of signal that this might continue - this sort of policy action might continue.

 

Dave Bittner: So to what degree should regular people be concerned about this, you know, for our listeners? Is this, you know, high-level espionage kind of thing or does it affect people in their day-to-day lives?

 

Sarah Graham: In terms of targeting of - with spyware, our report doesn't go into this in great detail. And I would really encourage listeners to go and check out some of the - some organizations that really give a lot of context and color to these sorts of targeted surveillance intrusions coming from organizations, like I mentioned, of Citizen Lab or Amnesty Tech. But that doesn't deny the fact that we all can take some personal steps in our personal digital footprint in securing that. What I think does matter, though, for most Americans is what I talked about with U.S. investments. When we dig into this a bit more in detail, we actually found that a few different pension funds, for example, are invested in spyware companies. If I am recalling correctly, I believe a pension fund out of New Jersey and Washington State are included in this. And, while that might not be something that an everyday person is aware of, being able to have some context over where your finances are being invested into is certainly a first step in understanding what's going on and how entangled these sorts of ecosystems actually are. [ Music ]

 

Dave Bittner: Ben, what do you think?

 

Ben Yelin: I guess I just didn't have a great understanding of the spyware market and how it's a very dynamic market.

 

Dave Bittner: Yeah.

 

Ben Yelin: You have resellers and brokers playing a more critical role than anybody had previously understood. So it's harder to track relationships between the users and the specific vendors. And there are a lot of countries that have come into the market as well. So a really interesting conversation.

 

Dave Bittner: Yeah. The big aha for me was the degree to which U.S. companies are investing in these spyware companies and I would have thought that was prohibited, but Sarah set me straight, sort of the - let me know that that perception was an error. Evidently, it's a common perception, but it is indeed not the case.

 

Ben Yelin: So it wasn't just us being ignorant.

 

Dave Bittner: No, no. Lots of people are ignorant when it comes to this. Evidently, it's a common misunderstanding. So I'm really glad that we had the conversation and she could set me straight and educate me. So, again, thanks to Sarah Graham from the Atlantic Council's Cyber Statecraft Initiative for joining us. Their research is titled "Mythical Beasts: Diving into the depths of the global spyware market." And we will have a link to that in the show notes. [ Music ] And that is "Caveat," brought to you by N2K CyberWire. We'd love to know what you think of this podcast. Your feedback ensures we deliver the insights that keep you a step ahead in the rapidly changing world of cybersecurity. If you like our show, please share a rating and review in your favorite podcast app. Please also fill out the survey in the show notes or send an email to caveat@n2k.com. This episode is produced by Liz Stokes. Our executive producer is Jennifer Eiben. The show is mixed by Tré Hester. Peter Kilpe is our publisher. I am Dave Bittner.

 

Ben Yelin: And I'm Ben Yelin.

 

Dave Bittner: Thanks for listening. [ Music ]