Android device eavesdropping investigation.

Dave Bittner: [00:00:03] Hello everyone, and welcome to the CyberWire's Research Saturday presented by the Hewlett Foundation's Cyber Initiative. I'm Dave Bittner, and this is our weekly conversation with researchers and analysts tracking down threats and vulnerabilities, and solving some of the hard problems of protecting ourselves in a rapidly evolving cyberspace. Thanks for joining us.

Dave Bittner: [00:00:26] And now, a moment to tell you about our sponsor, the Hewlett Foundation's Cyber Initiative. While government and industry focus on the latest cyber threats, we still need more institutions and individuals who take a longer view. They're the people who are helping to create the norms and policies that will keep us all safe in cyberspace. The Cyber Initiative supports a cyber policy field that offers thoughtful solutions to complex challenges for the benefit of societies around the world. Learn more at hewlett.org/cyber.

Dave Bittner: [00:01:02] And thanks also to our sponsor, Enveil, whose revolutionary ZeroReveal solution closes the last gap in data security: protecting data in use. It's the industry's first and only scalable commercial solution enabling data to remain encrypted throughout the entire processing lifecycle. Imagine being able to analyze, search, and perform calculations on sensitive data, all without ever decrypting anything. All without the risks of theft or inadvertent exposure. What was once only theoretical is now possible with Enveil. Learn more at enveil.com.

Christo Wilson: [00:01:41] There's this persistent rumor that our smartphones are surveilling us.

Dave Bittner: [00:01:46] That's Christo Wilson. He's an associate professor at Northeastern University in the computer science department. In a few moments we'll also hear from Elleen Pan. She's one of the coauthors of the research and a recent graduate of Northeastern University. The research we're discussing today is titled "Panoptispy: Characterizing Audio and Video Exfiltration from Android Applications."

Christo Wilson: [00:02:10] I don't know if it's clear where this started. What I would say is that this is probably a rational fear. These are very powerful devices. We know they have mics and cameras and that apps can use those sensors, and we also know that a lot of data is collected from these devices for the purposes of advertising. Now, this is why apps are free. So, the idea that the mic would be turned on, or that the camera would be turned on and, you know, information harvested for ad purposes it seems plausible.

Dave Bittner: [00:02:43] Yeah, and I think we also have these uncanny feelings that with all of this ad tracking that, you know, you'll find yourself browsing for something or shopping for something and then it seems as though that product follows you around on the Internet.

Christo Wilson: [00:02:56] Yeah, absolutely. So retargeted ads, the kind that you're describing, that are a specific product that follows you around - they're kind of creepy. And everyone has seen this.

Dave Bittner: [00:03:07] So, you decided to follow up on this. Take us through, what was your approach?

Elleen Pan: [00:03:12] We wanted to get a lot of apps and we also wanted to broadly cover what apps were doing in terms of different apps from different stores.

Dave Bittner: [00:03:21] That's Elleen Pan.

Elleen Pan: [00:03:22] We took 17,000 apps that were asking for camera and audio permissions, and then we ran automated experiments on them and collected the network traffic in order to see whether this type of media exfiltration was happening. And at the end of it, we ended up not finding audio being exfiltrated, but we did find some other stuff like screen recordings and some unexpected photo exfiltration as well.

Dave Bittner: [00:03:52] So, let's dig into it some here. Take me through the gathering up of the devices and the types of techniques you used to test them out.

Christo Wilson: [00:04:00] So, we have a closet full of Android phones - that's our automated testbed - and then we go to the various app stores and we grab the most popular applications from different categories. You know, as Elleen said, about 17,000 of them. So then, one at a time those apps get sent to a test phone, and there's a program that interacts with it. So it clicks on buttons, it types on the keyboard, it tries to access different screens. And that happens for about five minutes. And in the background, we're recording everything the app is doing. So everything that is sent over the network and the recipient of all of this information.

Christo Wilson: [00:04:41] So, most of the stuff that gets sent is just text. But what Elleen did is she developed a way to extract the video, and audio, and images that were potentially getting sent out. So anytime an app tried to send something, we would get a copy and we would know who sent it and who they were sending it to.

Dave Bittner: [00:05:00] And what kind of apps are we talking about here? Does it run the gamut?

Christo Wilson: [00:05:04] Yeah, so it's everything, you know, that's popular in the App Store across every category. So, all the top social networking apps. There's probably some games in there, note-taking apps, weather apps, Uber. Everything that's popular in the store, basically.

Dave Bittner: [00:05:21] I think that many people, their focus is on Facebook specifically, that they feel as though, you know, they've reached the conclusion that Facebook is listening in on them. The story that you hear is that, you know, I was discussing a vacation to Hawaii, and suddenly in Facebook up popped, you know, flights to Hawaii or, you know, travel sites, or things like that. Did you look specifically at Facebook?

Christo Wilson: [00:05:46] We did, but the caveat is that Facebook is easily one of the most complicated apps, probably in the entire app store.

Dave Bittner: [00:05:54] So, what were you able to do, and what were the conclusions that you came to?

Christo Wilson: [00:05:58] So, we tested Facebook the same way we tested everybody. We automatically run it and interact with it. But, for example, we didn't log in. So, if Facebook only records after you've signed in, we would not have triggered that.

Dave Bittner: [00:06:11] Now, did you do anything with playing audio files in a room to see if those audio files got then sent out?

Christo Wilson: [00:06:19] So, they were sitting in a room and there was some ambient activity. We also kind of preloaded the phones with content. So there were images, videos, and audio clips already kind of sitting there, and we were waiting to see if the app would notice those and try to send those as well.

Dave Bittner: [00:06:38] And so, when you're looking at the phones and the kinds of things that they're sending back, what specifically was being trafficked?

Elleen Pan: [00:06:45] So, in terms of all traffic, we saw a lot of text, as mentioned before, but with media specifically we saw some photos and videos. And a lot of the photos, they were intentionally sent, so it was part of the app's purpose to send these pictures home, based on, like, sharing or other intentional activity in the app. But we manually validated all the media that was sent out, to ensure the ones that we claimed were leaks were actually leaks.

Christo Wilson: [00:07:15] So, for example, there was a small number of apps that, say, they do photo editing, and the assumption from a user would be that this is local, right? You edit the photo and add filters on your phone. But actually what they're doing is they take the image and send it to a server. The processing is done remotely and then they send you the result back. This was either not disclosed or it was buried deep in a privacy policy where no person would ever see it.

Elleen Pan: [00:07:43] In addition to that, some of them don't have indication that they're using any type of Internet services, so, like, social media sharing or posting on a feed. And so it's really reasonable to expect that the user would have no idea that photo is leaving your device. And we also found some that were not encrypted over the Internet so eavesdroppers could basically take a look at the network and see other people's pictures.

Christo Wilson: [00:08:08] The other disturbing case study we found was this library that's targeted for developers to help debug their apps. So, this library was essentially doing screen recording. So, you would open an app, everything you did in that app would be streamed to a third party for analytics and debugging purposes. But this is of course not disclosed to the user. There's no indication that everything on the screen is being recorded and sent. It's equivalent to essentially someone looking over your shoulder and watching everything you do in this app.

Dave Bittner: [00:08:42] Now, were there any examples where the apps were clearly up to no good, that there was foul play going on?

Christo Wilson: [00:08:49] I would say no. We were initially sort of concerned that we were going to find really malicious apps, things that were spying on people, really invasive kinds of recording like people have found in the past with things like SilverPush. And we really didn't see that. You know, there were these kind of omissions where the app could have been clearer about its design. And then there was the screen recording which, you know, if I'm being honest, there's legitimate reasons why a developer would want some of that information to help debug. But again, it's creepy that it's not being disclosed, and it shouldn't be happening to everyone all the time.

Dave Bittner: [00:09:31] So, we often hear about, you know, I think the common example is, like, a flashlight app, that when you install it, it asks for permission to use your microphone and your camera, and access all of your data and all your contacts and, you know, the joke is, well, why would a flashlight app need to know all those sorts of things. But you did not run into that sort of thing where, for example, a flashlight app would be, you know, sending all of your personal information to a server halfway around the world.

Christo Wilson: [00:09:56] So there are definitely apps that are over-provisioned, right?The flashlight that asks for every conceivable data source on your phone. But you're right. That flashlight, and the other apps in our data set, we did not see them engaging the camera or the mic in a way that a user would not be expecting, you know, just to surveil them.

Dave Bittner: [00:10:19] Now, the work that you did, you were looking at Android only. Was there any look into anything on iOS?

Christo Wilson: [00:10:25] So, iOS is unfortunately complicated. We did not look at iOS. It's just much harder to do that kind of testing, because it's not open-source and it's very locked down. But, that said, the capabilities for the things that we saw, those capabilities are also available in iOS. So, for example, the library we found that was recording the screen. There's a version of that library for iOS. So, these things are almost certainly happening there on iPhone as well.

Dave Bittner: [00:10:54] All right. So you're able to extrapolate the likelihood that these sorts of things are happening on that side as well.

Christo Wilson: [00:11:00] Exactly.

Dave Bittner: [00:11:01] So, what is the future here for you? The conclusions that you've come to, and where do you go from here?

Christo Wilson: [00:11:07] So, one direction that we're definitely focused on is IoT. A very similar set of concerns, that you have this microphone sitting in your living room, or a smart TV, right? And it's watching you and listening to you. And do you really know where that data is going? Do you really know when it's listening? These are very legitimate concerns. We essentially have a studio apartment in our lab space that's an IoT lab. It's just full of devices and then we have, you know, boxes more after Prime Day coming. So, we're going to be looking at those very closely to see how they're recording, how they interact with each other, where that data is going, because that's the frontier.

Dave Bittner: [00:11:55] So for folks, in terms of protecting themselves, what are your recommendations?

Christo Wilson: [00:12:00] Yeah, so this is a tricky question because you have limited ability to stop apps from doing things once they're installed. I guess the high-level advice is just think twice before you install apps. Do you really need best, brightest, flashlight? Because any time you install these apps and grant them permission, even if the app itself isn't malicious it can come with third-party code that may be.

Dave Bittner: [00:12:28] Do you understand this impulse that people have, this reaction, the sense that they feel as though their phones are listening to them even if they might not be?

Christo Wilson: [00:12:37] Oh, absolutely. It doesn't help that companies like Facebook have these patent applications for systems that would do exactly this. And there have been cases in the past where apps were caught turning the mic on and surveilling without the user's consent or knowledge. An ad company called SilverPush was doing this. There was a recent case where a soccer app was engaging the mic to see if the user was around an illegal broadcast of a game. So these concerns are valid. Plus, you then have the targeted advertising issue which is very creepy. And the phone is always there. So when you talk about a vacation to Hawaii and then you get that ad for Hawaii, it doesn't seem like a coincidence.

Dave Bittner: [00:13:21] Right.

Christo Wilson: [00:13:22] It's very easy to infer that something happened.

Dave Bittner: [00:13:26] Even though it may not have happened the way that you're sort of connecting the dots in your mind.

Christo Wilson: [00:13:31] Yeah. So, one thing to think about is, is audio transcription technology really that accurate? You know, for anyone who's used Siri or Google Assistant, I think we all get the sense that they're not really that great yet. So the idea that the easiest way to get your data would be to surveil your audio - that's a bit of a stretch.

Christo Wilson: [00:13:55] The other thing is that we don't really realize how much data we give away routinely just through our actions online. Every time you're searching, every time you're browsing, every time you're clicking, all of that's being collected and then put through a machine learning algorithm is to infer things about you. If you're going on travel sites or you're searching for day trips in Hawaii, it's not hard to infer your interest in Hawaiian plane flights.

Dave Bittner: [00:14:25] Yeah, and even tracking your location, you know, an app like Facebook knows where you are and what events you might be interested in, and your other friends and your friends list who are at the same place at the same time. There's certainly a lot of things that they can put together without needing to listen to you.

Christo Wilson: [00:14:42] Absolutely. You may not be the one doing the searches for vacations let's say, but if your friend is, or your partner is, it's so easy to connect that to you.

Dave Bittner: [00:14:53] What's the bottom line here? For those who are out there with these concerns that primarily Facebook and many of the other apps may be listening to them, do you say, maybe not such a big concern?

Christo Wilson: [00:15:05] So I would say that, right now, this is not as big a concern as people think. The idea that you're constantly being surveilled by your phone is probably not true. Now, that doesn't mean there won't be apps in the future, or corner cases where apps are malicious, and this happens. But in general, sort of writ large, there's other things you should be more concerned about.

Dave Bittner: [00:15:28] Elleen, what's your take on this? What conclusions did you have?

Elleen Pan: [00:15:31] Yeah, I definitely feel the same way, that there is just a lot of ways that companies track us other than the microphone. And in that way, I just feel like I should be more mindful of my activity online, or just how I'm being tracked in those ways and not as specifically concerned about my phone listening to me, but just being aware of my privacy as a whole.

Christo Wilson: [00:15:57] One thing that's worth mentioning is, you know, in cases where we did find apps that were not disclosing things, right? We did responsible disclosure to those companies. We also responsibly disclosed all of this to Google. So, Google issued a statement saying that they've taken appropriate action against some of these apps, specifically ones that were recording the screen. So, this was a case where some direct good came out of the research.

Dave Bittner: [00:16:29] Our thanks to Christo Wilson and Elleen Pan from Northeastern University. The research is titled "Panoptispy: Characterizing Audio and Video Exfiltration from Android Applications." We'll have a link in the show notes of this episode.

Dave Bittner: [00:16:46] Thanks to the Hewlett Foundation's Cyber Initiative for sponsoring our show. You can learn more about them at hewlett.org/cyber.

Dave Bittner: [00:16:54] And thanks to Enveil for their sponsorship. You can find out how they're closing the last gap in data security at enveil.com.

Dave Bittner: [00:17:02] The CyberWire Research Saturday is proudly produced in Maryland out of the startup studios of DataTribe, where they're co-building the next generation of cybersecurity teams and technology. The coordinating producer is Jennifer Eiben. Editor is John Petrik. Technical editor is Chris Russell. Executive editor is Peter Kilpe. And I'm Dave Bittner. Thanks for listening.