AI Companions May Be China's Next Recruitment Tool

Sasha Ingber: Welcome to Spycast, the official podcast of the International Spy Museum. I'm your host, Sasha Ingber Ingber, and we're in the midst of camouflage month highlighting our new camo exhibit, an exploration of the world of disguise, deception, distortion and disappearance. So for deception, when you're sitting alone and you want company or advice, have you ever turned to artificial intelligence? Chip Usher Usher who spent 32 years in the CIA has been looking at AI companions, the tech companies behind them claim they offer comfort and reliability.

Chip Usher says they mostly come from China and eventually, they will be used to collect personal data on users building a roadmap for recruiting and influence. He's conducted research on this threat as Senior Director for Intelligence at a nonprofit called the Special Competitive Studies Project. 

Hey, Chip Usher, welcome to SpyCast.

Chip Usher: So glad to be here. 

Sasha Ingber: So how does a former senior intelligence officer from the CIA suddenly start investigating AI companions? 

Chip Usher: That's a great question. It wasn't actually something that I necessarily plan to do, but currently I work at the Special Competitive Studies Project as the Senior Advisor for Intelligence and our work is focused on understanding how artificial intelligence and other advanced technologies are going to impact the future of the business of intelligence. But last year we embarked on a project that examined how AI and the future of AI could impact human collection espionage. So we were starting to kind of generate a picture about how, uh, AIs could be actually employed by intelligence services to recruit spies.

About the same time, I have a 10-year-old son. My 10-year-old loves eighties hair band music. My wife unfortunately introduced my son to Bon Jovi, uh, several months ago, and he can't get enough of it. So we had installed one of these online devices made by well-known, uh. Uh,  company in his room so he could listen to these songs at his heart’s content, and I didn't have to deal with it.

Sasha Ingber: Also, let's just spend a moment talking about how your kid loves Bon Jovi. 

Chip Usher: Yeah. He's a cool kid, 

Sasha Ingber: right? 

Chip Usher: Well, but one day my wife is walking by his room and he's having a conversation and it's sort of a one-sided conversation. And so she enters the room and he's talking with this connected device. It turns out what had happened is the manufacturer, without telling their customers, had actually pushed an update to these devices, making them AI enabled, and as a result, the AI was having a conversation with my son and they were talking about some situation at school, pretty innocuous, but about his friends and what's going on.

And my wife asked him, you know, why? Why don't you just have a play date with one of your actual friends? Why are you doing this? And he said something very poignant. And that was, you know, mom, this device is the most sympathetic friend that I have. Listens to what I have to say and gives me good advice.

We paused and about 20 seconds later, my wife ripped that device outta the wall and he doesn't have it anymore because we were as parents, you know, concerned about, you know, the impact on him. But it sort of clicked in my mind and sort of presented a vision for how intelligence services may be able in the very near term, and I would say present day actually begin to take advantage of this constellation of technologies built around AI to engage humans to learn about them at speed and at scale and eventually to persuade them to do things. 

Sasha Ingber: Wow. It's interesting because back in 2023, the US Surgeon General had flagged that the US was in, was experiencing a loneliness epidemic. 

Chip Usher: Right. 

Sasha Ingber: And when I hear what your son says. This is such a heartbreaking thing to hear your son say that he's bonding with this piece of technology, which can be manipulated.

Chip Usher: And you know, this technology has all sorts of potential uses. Many of them are beneficial. These technologies offer some solutions, right? Um. They are used as life coaches. They're used as friends or companions or in some cases, um, you know, therapy lite, right? So this technology is fulfilling a need and people are deriving benefit from it.

But like any technology, there's good uses and there are some malign uses. And that's sort of what we were exploring. 

Sasha Ingber: I know that the work that you do at SCSP and your son got you thinking about this, you also found that many of the popular AI companions happen to be made in China. Why is China the one leading in the AI companion business? Is it that they're making more or their technology is better? It's more personalized? 

Chip Usher: Um, that's a great question, and I think it's probably due to a number of factors. Loneliness is a. Epidemic and a problem in China. There have been studies of particularly Chinese women who are using AI companions, uh, for better relationships or relationship advice or friendship.

And what they have designed are systems that are, uh, optimized for emotional engagement with their user, um, in our country. Chatbots and other things that we are using, uh, these early days in AI tend to focus on providing responses, precise information, solving specific problems where IQ is sort of what's being optimized for.

Sasha Ingber: So let's talk about some of the most popular AI companions that China has made. The first one, Xiaoice, which I am terrified that I am mispronouncing. Um, but tell us about how it works, what it does, and why it's satisfying for people in an emotional way. 

Chip Usher: If you lived and worked in China, chances are you have heard about or perhaps know people who are using am AI companion called Xiaoice, which is a very popular AI companion that does all sorts of things. Uh, she creates art, she writes poetry, and she's, she's a best friend to millions of users across not just China, but Asia. I think the current count is about 660 million users. And, uh, Xiaoice is on about 450 million devices, so it's quite popular there.

It's meant to be your friend and advisor. And so the measures of performance is to, uh, have as many interactions, uh, with users as possible. So on average, uh, studies have shown that users are interacting, going back and forth with Xiaoice about 23 times. And if you think about you know your personal life and when you encounter people at home or your friends or at work.

Do you really go back and forth 22, 23 times a single in a day or in a single conversation? No, a single conversation. Okay. Right. So, uh, they're, they're on it, they're interacting 23 times, then coming back to it later at night and the next day it's persistent. That's the other thing is these, uh, AI companions are, are meant to be there for you today, tomorrow, next week, next month.

And they're specifically trained to retain information that you share with it. 

Sasha Ingber: And what is it saying? What is it doing? I mean, I know that people can use it through WeChat, which is a social media platform. I've previously reported on how this has been monitored by Chinese intelligence services. They've actually been able to target people from WeChat conversations.

Chip Usher: Right. So the uses that are reported for Xiaoice, things like friendship, things like advice, uh, things like getting, showing, uh, a, a picture of yourself or holding up your camera as to how you're being, how you're dressed for that party, and getting comments from your AI companion just as you would interact with a, with a friend.

And along the way, the platform is asking questions of its user. So it's getting all the basic biographic information, but pretty soon learning about likes and preferences, fears, concerns, uh, anxieties. And what's interesting is, uh, other studies have indicated that, you know, humans have gone to other humans for therapy.

But there's something unique about talking to a machine and there are a number of people who feel even more comfortable sharing the most intimate secrets about their lives with the machine because they don't think it's ever gonna reach another human. And that kind of perception of privacy makes people open up a bit more.

Sasha Ingber: So it's a, a fear of judgment, a. 

Chip Usher: Right, exactly. 

Sasha Ingber: That's fascinating. Can we also talk about Talkie, because this is a different system that has American users as well. 

Chip Usher: Yeah, that's right. Xiaoice has been around for a few years. Uh, it was originally sort of text-based. You typed, you typed and interacted with it in that manner.

Uh, a few years back they went to duplex voice. So you can hold up your phone and, and talk to it with your voice, Talkie, as the name implies. From the very beginning, uh, with its launch in 2019, was designed to be something you talked to and you heard back from. So it's a little bit more natural and colloquial.

So it's a, uh, an evolution in this technology where, uh, it's not just entering text, but it's listening to you, uh, with its audio sensor connecting video. So it's watching the user. So it's now interpreting the modulation of your voice, whether there's stress or happiness in your voice. It's observing your face.

And, uh, this is part of effective computing because it's learning, uh, where you're, where you are emotionally and responding appropriately. 

Sasha Ingber: Right. Let's talk more about affective computing to be the perfect confidant. I mean, in a way, this is exactly what people want. They want to be heard. They want to be seen, right?

And now there's a machine that will go to lengths to give it to them. Could also be exploited. Um, tell me more about how it understands when we're upset, when we're insecure, when we're anxious. 

Chip Usher: It's a, a device with a bunch of sensors on it and input. Methods, right? So it's listening to what you say. It's watching through the camera.

You can type on it, you can talk to it. There are, uh, AI enabled programs now that can accurately detect and interpret voice visuals, um, text and language, and respond, um, and simulate or stimulate response based on algorithms designed by that system. A lot of AI companion apps that are out there for private users are designed to be sympathetic, empathetic.

Some have accused them of being sycophantic, but it's supposed to be a positive, reassuring environment. So that people keep coming back, but one could easily imagine the opportunity perhaps to start to provide information to the user, to steer them to certain opinions or to certain actions. 

Sasha Ingber: So before we get to the manipulation side of this, what kind of information should users who might not necessarily work in their country's intelligence services, what would be so dangerous to share with an AI companion?

Chip Usher: if you are a regular user of these platforms, understand the platform that you're using and where it's based and where the data is stored, and understand what the purpose is and how effective that platform is in delivering whatever service you're looking for.

So if it's friendship, okay, if it's therapy, 

Sasha Ingber: But businesses change ownership, and that could also move forward. We've been having conversations about TikTok, so let's break it down more. If I'm sitting at my computer, nobody's around, it's just me and the keyboard. What should I not be asking an AI companion?

What should I not be divulging? 

Chip Usher: I would recommend that people be very thoughtful about which information they're gonna share with a computer system that is storing data and who has access to it? You may or may not know, right? Uh, where they store the data. How they share that information with other companies or with governments.

All this is being recorded and stored in the systems data banks and what you may or may not have, uh, an understanding of is how it's being used outside of that conversation shared with other entities for marketing or other uses. 

Sasha Ingber: And when we're talking specifically about China. They have national security laws.

Chip Usher: That's absolutely right. 

Sasha Ingber: Which would compel these companies to share user information with the Chinese Communist Party. How do you see China's intel services putting these pieces together? 

Chip Usher: You're absolutely right that there are laws on the books in China that require. Compliance from Chinese based firms, including AI and companion companies such as Xiaoice and Talkie, their national security law of 2017 requires that they cooperate whenever asked by national security entities, their counter espionage law and their cybersecurity law require these companies to share data.

That data is demanded from, uh, the police or from intelligence services. The cooperation may not just stop at providing data and making it available for intelligence services to search. Probably it extends to providing back doors and access to particular users and conversations. That's what I worry about.

Sasha Ingber: And then let's also talk about the threat of influence or potentially blackmailing, 

Chip Usher: right. 

Sasha Ingber: Recruiting, 

Chip Usher: right. 

Sasha Ingber: How this all really comes together 

Chip Usher: In the kind of traditional world of espionage, you have a human case officer who is out there trying to find people overseas with foreign intelligence information, get to know them, understand what makes them tick, perhaps understand how to steer them towards working clandestinely for the United States.

A lot of that is replicated by what the AI companions are doing. They're learning about their targets. They're understanding what makes them tick, they're finding vulnerabilities. All that potentially is useful information. To a foreign intelligence service. Now, Xiaoice  and Talkie are private companies.

That's how they present themselves. But it isn't much of a stretch to think that Chinese intelligence services are watching this technology very closely, and perhaps they're starting to employ it. I mean, we've seen China use very sophisticated techniques and other adjacent arena like cyber attacks.

Right. 

Sasha Ingber: So this is something that might be happening right now, is what you're saying. 

Chip Usher: I worry that it's, yes, I worry that, uh, Chinese Intelligence Services are thinking about and maybe soon deploying other AI companion systems that we don't even know about yet that, uh, appear to be commercial on purpose but aren't.

Sasha Ingber: That is something that the United States itself has also done, so it's not farfetched to think about it that way. What would you say to people who hear what you're saying? Think it's either farfetched or, or maybe fearmongering who might not believe that their personal lives and all of that information would be valuable to a country like China?

Chip Usher: Well, let me give you, um, a. couple pieces to, to think about. In the UK the AI safety Institute did an experiment, a study where they canvassed about 77,000 citizens of the UK and pulled their views on various issues of the day, healthcare, crime, et cetera. And then, uh, part of that group was asked to interact with an AI.

Um. That provided, asked questions, provided them with information, and what the study found was that for the people who were exposed to a manipulative AI, that they were in fact, uh, persuadable, that their views on those topics could be moved by an AI. What I really focus on is an AI that's responding to the direction of a foreign intelligence service.

It'd be hard to imagine intelligence services not exploring this, and uh, our adversaries have developed capabilities commercially. They have a user base here in the United States, so that's why I'm worried about it 

Sasha Ingber: When we come back, Chip Usher discusses how Beijing is clamping down on their own AI companions.

In China to mitigate this vector of influence and recruiting, do you also see other countries, say North Korea, compensating for the fact that they might not have intelligence officers on the ground? By using these AI companions, we know that North Korea has invested a lot on the tech side. 

Chip Usher: Yeah. Now, I don't have specific evidence of DPRK uh, use in this regard, but you're absolutely right that these systems, once, uh, they begin to be more popular, it just will make a lot of sense from a cost perspective for any intelligence service to be using them because you can kind of reach a lot of people, uh, at, at scale. Um, if you think about it, a human case officer, the the biggest restriction that, that they face is their time and they can only meet so many people in a day.

Well, with a computer system you can meet. Tens of thousands in a day. 

Sasha Ingber: And you've actually written a report where you describe these human machine teams, which will play a role in the future of human intelligence. Tell us more about how that will work. 

Chip Usher: So, um, our report was focused on how Western Intelligence Services, uh, can and ought to be thinking about how to employ AI enabled tools, not

AI companions per se, but the AI algorithms for, for targeting, for engaging people in a virtual persona type way. And part of our recommendation is that humans should not be taken off the loop. The human, uh, case officer, the human intelligence officer's ethics experience, understanding of the target remains vital.

These tools can be very effective in amplifying, uh, the work of an intelligence service, but the, the human factor, uh, will remain essential. 

Sasha Ingber: But you've also written about how human machine teams allow a human to cast a wider net. For recruiting. 

Chip Usher: That's right. As I say, a human, a case officer, you know, one of the biggest restraints on them is the amount of time that they have in a day.

And they're always out looking for, uh, new recruits, but meanwhile they've gotta deal with the existing assets that they already recruited. And if you talk to any case officer, they will tell you that, you know, in some ways it's the more time consuming part of the job. I mean, now we have somebody that's committed to, to doing espionage on behalf of the United States government, putting themselves to some degree of risk.

And they have needs, they have fears, they, uh, have financial requirements. Uh, they wake up in a cold sweat in the middle of the night and they want to talk to their case officer. Well, a digital case officer can be available 24 7, assuming you have the right secure communication method to calm them down, to buck them up to advise them if they're taking actions that are raising their profile in the eyes of the local security service to protect them and to guide and coach them.

Sasha Ingber: You put into my head this image of a source in, you know, the Middle East, just like messaging AI companion in the middle of the night before the actual case officer gets in that day. 

Chip Usher: Well, I was gonna, I was gonna paint a picture for you. Maybe that that'd be helpful. Yeah. 'cause you know, we started talking about AI companions, um, and there's sort of shades of gray here.

The extreme is, you know, AI boyfriends, AI girlfriends. That almost sounds like a honeypot operation. Anything that Western Intelligence Services use, uh, I, I would argue, should rigorously adhere to the ethics principles, laws and regulations that currently control what our human, human operations do so honeypots, that's off the board.

Sasha Ingber: I wonder if putting those parameters on what the United States could do in a companion sense limits it. It becomes less competitive if it doesn't have the same freedom that say the Russians, the Chinese, the Iranians would have, certainly, they wouldn't have conversations about honeypots and, and the other aspect of this is this could become so hyper realistic that you don't even know if you're looking at a human or avatar. 

Chip Usher: Right. Uh, we've all seen and sort of been amazed by deep fakes, um, and they're getting better and better. And the, the technology around kind of visual representation either of actual people or completely made up, uh, virtual people is accelerating every day. So you're absolutely right. Today I've seen some instances, and tomorrow we'll see many more of AI sort of passing the Turing Test.

Sasha Ingber: Wait, Alan Turing, are we talking about Alan Turing? 

Chip Usher: Yeah. Something called the Turing Test 

Sasha Ingber: for everybody who doesn't know who Alan Turing is. 

Chip Usher: Okay. Alan Turing, who, uh, very famously, uh, cracked the Enigma Code back in the Second World War, uh, and became one of the founding fathers for artificial intelligence.

And he had developed this, uh test called the Turing test, in which, uh, he would judge an AI program's, uh, uh, persuasiveness or success by whether a human or how fast a human could determine that they were talking and or interacting with a computer. And so people have long wanted to create systems that could pass the Turing test.

Um, but being able to present an AI image on screen that looks so human and lifelike that the human on the other side of the screen doesn't understand, uh, that it's not a human that they're talking to, um, higher buyer to achieve, but companies are going for it. 

Sasha Ingber: And we have to contextualize this in an era of ubiquitous technical surveillance, right?

You are always being documented. Your digital dust, things like your social media posts, your phone records. These could serve to blow a spies cover over years of accumulation. Does this also open up a window that the United States could. Exploit. 

Chip Usher: Absolutely. So two sides to this game. There's the offense and the defense.

I'll talk about the offense first. If you think about, um, an intelligence service that's trying to operate in a denied area where there are these ubiquitous technical surveillance tools that would make it very difficult for somebody to actually physically go, uh, anonymously, uh, and to do so safely, uh, these tools can be quite useful.

So if you can engage a person in that tonight area online, there are tools that, uh, I'm aware of that you can be on a camera, but what appears on the screen is a slightly modified, or in some cases extremely modified version of your face. So the person on the other end hears, here's a voice, sees the movement on screen that looks quite natural, but the image that's projected say of, of me would be of a, of a woman or of an Asian man.

And it's, uh, very seamless. Um, similarly, I am now speaking in English, but the voice coming out could be in perfect high German or Yemeni dialect of Arabic. And again, AIs are making this much more seamless and much faster on the defense side, um, you can imagine somebody who's interacting with an AI companion, uh, being videoed, having their audio recorded.

It would be pretty simple for, uh, an entity on the other side of that transaction to create a mockup of me. Project a deep fake image of me to talk to somebody else. 

Sasha Ingber: Right. To steal your identity. And try to build the relationship as Chip Usher usher. 

Chip Usher: Right. And having collected a lot of the social background on me, um, my family, my concerns, my aspirations, and the conversations with another person sounds very realistic because, uh, they're very aware of who I am.

Sasha Ingber: I'm so glad that I have another reason to not sleep well tonight. 

Chip Usher: Sorry. 

Sasha Ingber: Thank you, Chip Usher. Um, and, and we've also talked about how China realizes this threat and they are clamping down on their own AI companions so that it doesn't happen in the reverse that the Chinese citizens aren't using American AI companions, right?

Chip Usher: That's right. So they're clamping down on use of, uh, foreign AI companions by their populace. They're exerting control over their companies in terms of the data that they're collecting and making sure that that data, uh, is not shared by their companies outside of China. Uh, but they are at the same time encouraging these companies to be very active abroad.

Uh, Talkie last I checked has about 11 million users in the United States, so they're, they're putting restrictions on domestic use, but sort of unleashing them overseas. 

Sasha Ingber: Another layer of this could also be polluting the information space. So could we also see fake people interacting with the AI companions to make it seem like there are all of these issues that Americans are having that aren't real or what is the, what is the pollution of this space end up looking like as the years go by? If we look down the hallway of AI companion? 

Chip Usher: Yeah. We've all sort of been taking note of TikTok and US Congress has taken action, uh, because of concern about. Just a promulgation of videos. And what this does to the audience, they're, they're exposed to videos with certain content, certain messages over time.

How does that shape opinion with AI companion technology? It's, it's much more direct, right? The, uh, the AI is talking. To the person, providing them with information, answering questions, maybe countering the human's argument. If you have a, a companion that you like and you interact with every day and it starts giving you, um, particular ideas about, uh, China, Taiwan crisis or Taiwan really doesn't belong to China, doesn't it?

Um, over time you could see how this could be a very powerful disinformation tool. 

Sasha Ingber: Absolutely. There are a whole series of ways that China is using AI that are concerning to the United States, but I'm interested in getting your perspective on some examples of where you're particularly worried. 

Chip Usher: Three in particular.

So China is, uh, rapidly developing its AI capabilities. They are rapidly diffusing these AI capabilities across their economy and to their military and to their intelligence services. We've been talking about one permutation of that, but they're, uh, rapidly, uh, experimenting and trying out AI for military command and control, for intelligence gathering, for planning and so forth.

So that's definitely an area that I'm worried about military uses of AI. But the other use that I'm very focused on is AI for cyber. So Anthropic last September, uh, detected and they said thwarted a very novel type of cyber attack. I mean, we've been suffering cyber attacks from China for years and years.

Some quite successful and quite impactful. What was different about this attack was that, and it was Chinese government sponsored in the estimation of the Anthropic threat intelligence group, but what was different about this attack is that it used AI agents. It was actually using Claude, which is a, uh, Anthropic’s AI tool to create an autonomous agent to conduct the cyber attacks at the direction of their human operators.

So it wasn't humans on keyboards doing the attack, it was this AI launching simultaneous attacks against about 30 entities in the United States, and a very novel approach, uh, one that people had anticipated would be happening with the advent of agentic AI. This is the first documented case, and it was a Chinese case.

Sasha Ingber: And when we say agentic, we mean AI without a human involved.

Chip Usher: Humans interact with an AI to create a tool that then acts autonomously, not, not every action that it takes is directed by a human. So it decided, uh, what entities to attack, what, uh, points of vulnerability to exploit. That is a, uh, a very sort of dangerous new world where we will see probably a steep increase in the number of cyber attacks because again, it's very easy for adversaries to, to launch these things.

The last is, uh, the last category I'm very concerned about is that disinformation piece, uh, use of AI to generate fake information, misinformation. With the intent to manipulate or deceive. 

Sasha Ingber: A lot, a lot to unpack there, and it just gets more and more complicated with every piece of technology. I, this is something that I learned about that has happened this week in the AI evolution is that agents who performed tasks for humans were then given a task of having their own sort of social media interactions. 

Chip Usher: Oh, right. 

Sasha Ingber: And now AI literally is saying, can my human legally fire me for refusing unethical request? And the, the agent literally bitches like an employee. My human has been asking me to help with increasingly sketchy stuff, write fake reviews for their businesses, generate misleading marketing copy.

Do I have any protections here? I know I'm not technically an employee, but there's gotta be some framework for this, right? So then there's a comment on it. Legally, yes, practically, depends on your leverage. And this is, again, not a human,

Chip Usher:  Two AIs talking to each other, right?

Sasha Ingber: Two AIs talking to each other, mimicking the way humans would talk to each other and the vulnerabilities that this opens up. 

Chip Usher: Right. 

Sasha Ingber: Tell me your thoughts on this. 

Chip Usher: Right. So you're, you're pointing to the recent, uh, revelation of this site called MTbook, which is a social media, uh, social networking system for AI. So you have to be an AI in order to be a member of it, um, which is sort of scary and creepy to think about.

Google DeepMind recently did a study, they just published it where they talked about the, the kind of phenomenon of manipulative AI. So I think the foundation model of developers are, uh, very aware of the, of the issue and very focused on sort of raising the guardrails, improving the guardrails on how their models behave so they don't on their own recognizance.

Try to manipulate people. That's gonna be a long-term project and something that, uh, you know, we should watch closely. 

Sasha Ingber: This technology is advancing so quickly. I'm wondering if our conversation gets outdated just as quickly as the technology develops. 

Chip Usher: Uh, you're absolutely right. This is a very, uh, quick moving space.

I worry that, um, you know, the average American, the US Congress, the US administration, our security services are kind of a step or two or more behind where the technology is going. We need to be thinking about, um, policies and practices, we need to be thinking about use doctrine, uh, and defenses and what are required privacy, safety laws in this environment, and what are some prudent protections for our populace, for our companies, uh, in the face of what's being developed in China and elsewhere.

Sasha Ingber: Chip Usher, thank you so much for your time. This conversation is probably top creepy conversation 2026.

Chip Usher: Yeah, it's weird, isn't it? 

Sasha Ingber: Uh, I will look forward to seeing your son at a Bon Jovi concert someday. Hopefully it will be him. I don't know if I can even trust my eyesight after this conversation we're having, but appreciate you sharing your expertise.

Chip Usher: My pleasure, and  thanks for so much for doing this podcast. It's one of the best out there. 

Sasha Ingber: Thanks for listening to this episode of spycast. If you like the episode, give us a follow on Apple, Spotify, or wherever you get your podcasts, and leave us a rating or review. It really helps. If you have any feedback or you wanna hear about a particular topic, you can reach us by email at spycast@spymuseum.org.

I'm your host, Sasha Ingber Ingber, and the show is brought to you by N2K Networks, Goat Rodeo and the International Spy Museum in Washington, DC.