Difficulties and biases of online proctored exams.

Lauren Daming: I think we might see it change kind of the landscape of how exams are given and how we do that part of education.

Dave Bittner: Hello, everyone. And welcome to "Caveat," the CyberWire's privacy, surveillance law and policy podcast. I'm Dave Bittner. And joining me is my co-host Ben Yelin from the University of Maryland Center for Health and Homeland Security. Hello, Ben. 

Ben Yelin: Hello, Dave. 

Dave Bittner: On this week's show, Ben examines an FTC memo addressing potential Facebook antitrust claims. I look at an Ohio law that could make videotaping the police a crime. And later in the show, my conversation with Lauren Daming from Greensfelder, Hemker & Gale on privacy issues, litigation and best practices concerning universities' use of remote proctoring platforms for exams. While this show covers legal topics and Ben is a lawyer, the views expressed do not constitute legal advice. For official legal advice on any of the topics we cover, please, contact your attorney. All right. Ben, we got some good stories this week. But before we jump into all that, it wouldn't be the "Caveat" podcast if we didn't have a follow-up on the story about the Baltimore spy plane. 

Ben Yelin: An epitaph, if you will. Yeah. 

Dave Bittner: (Laughter) That's right. That's right. So we'll just do some quick follow-up here, put a button on this one. What's the latest? 

Ben Yelin: So we got a decision from the full Fourth Circuit U.S. Court of Appeals, the federal circuit here in the mid-Atlantic, ruling against the city of Baltimore and its use of that spy plane. It was actually an eight-seven decision, so closer than I would have expected. So it made permanent an injunction against the program. It's sort of moot because the new mayor elected in Baltimore City last year was against the program. It's been discontinued, defunded. It is now defunct. But the court not only argued that it was likely unconstitutional under the Fourth Amendment to have the spy plane taking real-time photos, but also that any information that was gleaned while that spy plane was operative - and it was for much of 2020 - those are probably not going to be admissible in criminal cases. So anything that's gleaned from that surveillance is going to be fruit of the poisonous tree and probably won't be allowed to be introduced at trial. The decision didn't say that explicitly. But it made clear that this is a very intrusive search. It violates a lot of Fourth Amendment principles. While defense attorneys and prosecutors will argue about this in court, it's unlikely that a conviction could be sustained if the only evidence available came from that overhead aerial surveillance. And I'll note, one thing that's interesting here is usually it takes a long time from when a new surveillance program is introduced to when the legal system weighs in. This is kind of an exception to that. The case moved rather quickly. And this was sort of a test case, the use of the aerial surveillance method in Baltimore City. And I think this will dissuade other municipalities, even during this time of heightened violent crime, from using this technology if, you know, you have a prominent appeals court saying that it's an unconstitutional search under the Fourth Amendment. 

Dave Bittner: All right. So the eye in the sky DVR has been unplugged for now (laughter). 

Ben Yelin: Yes. For those of you in Baltimore City, if you are still hearing a buzzing sound, check with your doctor because... 

Dave Bittner: (Laughter). 

Ben Yelin: ...The plane's no longer there. 

Dave Bittner: Right. Right. It's probably a police helicopter. 

Ben Yelin: Yeah. 

Dave Bittner: Yeah. (Laughter). All right. Well, let's move on to our stories. Ben, why don't you start things off for us. 

Ben Yelin: So a big blow to critics of big tech and those who want to see the big tech companies - the Facebooks, Googles of the world - broken up for antitrust purposes, and that's because a district court judge in Washington, D.C., dismissed two antitrust cases against Facebook. One of them was brought by the Federal Trade Commission, so that federal agency. And the other one was a suit brought by the states. About 46 attorneys general signed on to the suit alleging illegal practices, violating antitrust principles on the part of Facebook. The decision in this case is multifaceted. Part of the lawsuit is that Facebook acquiring Instagram and WhatsApp sometime in the past decade squashed competition in social networking. And then the other is, under Section 2 of the Sherman Act, the allegation is that Facebook corners too much of the market for social networking. Therefore, it's a monopoly. And our judicial system has the right to break them up. The reason that this judge dismissed the cases here is that the FTC in particular didn't properly allege that Facebook had the market cornered on this type of social networking. So the FTC came in. And kind of based on a - guesstimate, I guess, is putting it charitably... 

Dave Bittner: (Laughter). 

Ben Yelin: ...They said that Facebook controls about 60% of the market share for social networking. That kind of 60% estimate would have been really persuasive in a normal monopoly case, a products case. So if one company controlled, you know, 60% of the market on lawnmowers, you'd have a reasonable antitrust claim there. What's problematic here is the FTC didn't do a good enough job defining exactly what the market is. What counts as the type of social network that Facebook is? How do we know that they're cornering that part of the market if we can't even define that market? And it's the responsibility under the law of FTC to properly define that market in order for the federal court to take up an antitrust case. So what the judge here is saying is, you didn't do well enough on your first pleading here. You haven't properly defined that market. The 60% number is of no use to us right now. You have 30 days to go back and amend your complaint and make a proper allegation. Define exactly what the market is. And come up with some sort of persuasive evidence that Facebook actually does control 60% of that market. The judge said - and I would agree with the judge here - that's going to be very difficult for the FTC to do, not only within such a short time frame, but I think just generally. It is very nebulous what counts as a social network. And there are a lot of things that Facebook does that aren't, you know, strictly related to social networking. So do all of those things, you know - news feed curation, for example, does that count as part of that definition, because that really is a unique service? So those are going to be really difficult questions to answer. I don't think the FTC is going to come up with an adequate explanation within the next 30 days. And, frankly, I think this is a huge win for Facebook. They argued that there really is robust competition in the social networking market. They pointed to the rise of TikTok, which I think is kind of persuasive. TikTok is what the kids use these days. 

Dave Bittner: Right. Yeah. I mean, isn't - that's an interesting part of the argument as well, that Facebook- well, I guess I should ask you. Is it an interesting part of the argument that Facebook has a particular demographic? You know, as my kids would say, the olds (ph) use Facebook, right? 

Ben Yelin: Yes. 

Dave Bittner: (Laughter) The kids aren't using Facebook. 

Ben Yelin: Well, it does mean that there is a viable alternative for this type of social networking. And especially without properly defining what Facebook does, you could clearly make the case that for a given social networking post, there are a lot of opportunities to use different platforms. 

Dave Bittner: Right. 

Ben Yelin: So TikTok is a great example. If I have a video that I want to go viral, I don't have to post it on Facebook. There are videos that have gone viral on TikTok. There are pictures that have gone viral because they were originally posted on Snapchat. There are tweets that have gone viral. 

Dave Bittner: Right. 

Ben Yelin: So what service is Facebook specifically providing where it has a monopoly? I mean, it seems what the judge is saying is that they didn't properly define that service. And without properly defining that service, they can't make a determination that a monopoly exists. So I think this puts Facebook on pretty firm ground. I think people at Zuckerberg headquarters in Silicon Valley are probably very pleased with this case. I think there was a lot of hope based on the fact that, you know, we've seen members of Congress from both political parties and a lot of legal scholars saying there might be a viable antitrust case against some of these big tech companies. We might be able to break them up under the Sherman Act. I think this is just a splash - I don't want to say a splash of cold water. It's more like a splash of ice water. It's the ice bucket challenge of... 

Dave Bittner: (Laughter). 

Ben Yelin: ...Rejections of cases just because there was a lot of hope that we'd finally get some antitrust enforcement. And what the judge here is saying is, you know, you're not even really close. 

Dave Bittner: Yeah. I've seen a couple of responses to this. And one of them was, I think, from some legislators saying that this is an indication that Congress needs to go at it here and provide better tools to go after companies like this - that in the digital age, the Sherman Act needs to be updated, that it doesn't provide the proper tools to go after these sorts of companies. 

Ben Yelin: I think that's absolutely right. And there is an effort underway in Congress on a bipartisan basis. There are a series of bills - they've already been considered in committee in the House of Representatives - that try to amend portions of the Sherman Act to make them more relevant to breaking up big tech companies. The problem is it's going to be really hard to come to an agreement between the two parties on the exact parameters of those changes. They both agree that big tech needs to be broken up. I don't think they agree on the reasons why... 

Dave Bittner: Right (laughter). 

Ben Yelin: ...Why big tech needs to be broken up... 

Dave Bittner: Right, right, right. Right, right. 

Ben Yelin: ...Or, you know, the exact legislative changes. 

Dave Bittner: Right. Right. It reminds me of, you know, back in the days - the early days of Obamacare, when you'd see people say that, you know, 70% of people are against Obamacare... 

Ben Yelin: Right. 

Dave Bittner: ...And half of them didn't like it because they didn't like it. And half of them didn't like it because they didn't think it went far enough, right (laughter)? 

Ben Yelin: Yeah. Exactly. Exactly. I mean, it's just really hard to get Congress to agree on anything. 

Dave Bittner: Right (laughter). 

Ben Yelin: And so are they going to, in a, you know, relatively short time period, come up with a new antitrust regime that applies to big tech companies, especially when Congress is so narrowly divided? I certainly have my doubts. There are a couple of avenues of potential hope here. The states' case seems to potentially be maybe stronger than the FTC's case. The states could appeal the dismissal of that lawsuit. And, you know, the FTC, if they got some really good lawyers in a room for 30 days, maybe they can find a way to better articulate what that market is... 

Dave Bittner: Yeah. 

Ben Yelin: ...And add more specificity on why Facebook controls 60%-plus of that market. 

Dave Bittner: Yeah. There was an article in Wired that - they spoke to some folks about this who were hopefully making the case that this provided a roadmap for them to do just that. 

Ben Yelin: Yeah, I think it does. But I think what the judge made clear is, like, this isn't about going back and dotting the I's and crossing the T's and doing some editing, that they actually fell significantly short of the type of pleading they'd need to allow this case to move forward. And when you have a dismissal, it doesn't just mean that you've lost the case. It means you haven't properly alleged legal wrongdoing that would allow the court to consider the case in the first place. Maybe I'm being overly pessimistic from the point of somebody who cares about antitrust. But we'll see if the FTC can pull something together here. 

Dave Bittner: Well, we will keep an eye on that one as it plays out, for sure. My story this week comes from News 5 Cleveland, which is one of Cleveland's local... 

Ben Yelin: One of your go-to websites, right? Yeah. 

Dave Bittner: Well (laughter), it came by on my Twitter feed and attracted my attention, as you'll see why in a moment here. And it's titled "Proposed Law Making Cell Phone Video Of Cops A Crime Moves Forward By Ohio Legislators." Now, as is often the case, the headline is a little more breathless than the reality here. But (laughter) I still think it's something worth our attention. There's a proposed state law in Ohio that basically expands obstruction of justice laws. And what it does is it would include failure to follow a lawful order from police or diverting a law enforcement officer's attention. Those would be rolled into obstruction of justice. And so what folks are saying is that if someone standing by, videotaping some - you know, police going about their business, whatever that may be, that could be, I suppose, diverting a law enforcement officer's attention. That phrase makes me raise my eyebrows because, I mean, that could be anything. 

Ben Yelin: Yeah. 

Dave Bittner: Right? 

Ben Yelin: It is very subjective. So generally, one does have a First Amendment right to videotape the police in the course of their public conduct. 

Dave Bittner: Right. 

Ben Yelin: You can't peer into their house and just 'cause they're police officers and spy on, you know, their TV-watching habits. 

Dave Bittner: Right. 

Ben Yelin: But you generally do have a constitutional right to observe them and the execution of their public duties. 

Dave Bittner: Yeah. 

Ben Yelin: There are some complications there. So courts have made it clear that the police have the right, in the name of maintaining order and, you know, effectuating arrests, et cetera, to tell members of the public to move out of the way, basically. 

Dave Bittner: To back up, back up. 

Ben Yelin: To back up and clear the area. 

Dave Bittner: Right. 

Ben Yelin: Now, they're technically not supposed to do that simply because somebody is videotaping. It probably does happen where they tell people to clear an area, and perhaps that's done when the real purpose was to stop somebody from videotaping. But that's really hard to adjudicate in each individual circumstance. 

Dave Bittner: Right, right. 

Ben Yelin: So I think the part of this law that's not as problematic is obstruction of justice if a person keeps videotaping after they've been told to disperse. I think the part of the law that is potentially problematic from a First Amendment perspective is the part that talks about if the police officer is being distracted, because that is something that's kind of subjective. And, yes, it can be distracting if somebody is simply - you know, from a subjective point of view, from an officer's point of view, it probably is distracting to have somebody with their iPhone out, taking a video of your conduct. But, you know, is that objectively distracting? You know, would a reasonable police officer be distracted by that? I think courts might have a hard time coming to that conclusion. 

Dave Bittner: Yeah. 

Ben Yelin: So that might end up being a type of First Amendment violation because it's overbroad. So I think if this - and it seems like this law is on track for passage. I think if this law does pass, the portion of the law that's going to be problematic is this element about the police officer being distracted by somebody taking a camera. And I think we'd be remiss to not talk about the context here. You know, you and I followed the trial of Officer Derek Chauvin in Minneapolis. This became a case of worldwide notoriety because a bystander took a video. 

Dave Bittner: Right. 

Ben Yelin: And part of Chauvin's defense in that case - which was not successful, he was convicted - is that the fact that there were members of the public taking video during the incident - that it was distracting, and it kept him from applying the standards of his profession. 

Dave Bittner: Right. 

Ben Yelin: And the court and the jury rejected that argument. 

Dave Bittner: Yeah. 

Ben Yelin: But I think that that's really important context here, is I don't think we want to get into a situation where we're overly reliant on the police saying that they were distracted as a pretext to stop people from recording police interactions. 

Dave Bittner: Yeah, I think it's chilling. And hopefully, it'll play out quickly, and we'll get some resolution on it. But that one gives me pause for sure. 

Ben Yelin: Absolutely. Yeah. If, as expected, this law does pass, then I'm sure you're going to see some immediate legal challenges, a demand for an injunction probably from the ACLU in Ohio. 

Dave Bittner: The usual suspects, right? 

Ben Yelin: Absolutely. So we will keep our eye on whether that develops into a case. 

Dave Bittner: All right. Well, those are our stories for this week. We would love to hear from you. If you have a question for us or a story you'd like us to look at, you can call us. Our number is 410-618-3720. You can leave a message there, or you can send us email to caveat@thecyberwire.com. All right, Ben, I recently had the pleasure of speaking with Lauren Daming. She is from the law firm Greensfelder, Hemker and Gale, and we were discussing these proctoring platforms that professors are using for exams. I know this is something up your alley as an instructor yourself. Here's my conversation with Lauren Daming. 

Lauren Daming: This was a technology that existed before the pandemic, and when universities and schools across the country were forced to do remote learning, you know, it really created the potential for a need to have some sort of way to curb cheating when students were taking their exams at home. There's a couple different ways that these technologies work when they're looking into how students are performing on exams at home. The first is there's a type of live monitoring that some of the softwares used where an actual human is looking at video footage of a student taking an exam and taking note of their movements, their eye movements, looking for any type of indication that the student is cheating. And the other type of monitoring is more of an A.I.-based monitoring where the same type of video surveillance is being analyzed through some sort of algorithm that's picking up on the same type of eye movements, physical movements to see if a student is possibly cheating on an exam. And some of these softwares also have other credential identification or, you know, identity authentication measures that are used. They might ask a student to show their driver's license or some type of ID. They might ask the student to show photos or videos of the room where they're taking an exam so the university or the professor can assess whether the student is cheating or might be looking at notes or a cell phone or something during the exam. 

Dave Bittner: So what are the issues that we run into here when folks are using this technology? 

Lauren Daming: Well, there are quite a few. It's really, really quite a web of privacy issues here. One of the most notable ones is the potential for discrimination. I think we all know that AI software is kind of notoriously biased and often has trouble authenticating or recognizing, you know, students of color. So there's, you know, a potential for race discrimination. 

Lauren Daming: But beyond that, this type of software is really - you know, it's capturing somebody's physical appearance and looking into their homes and their living situations. They might reveal a student's, you know, clothing that they're wearing that might indicate they practice a certain religion or, you know, other symbols or objects in a person's home that might indicate a religious affiliation. And, you know, that always introduces the potential for discrimination. 

Lauren Daming: You know, I think another problematic aspect is the potential for disability discrimination. These softwares and even the live monitoring, they're looking for, you know, certain eye movements or physical movements that indicate cheating. And some of those that might be flagged as problematic might be movements that are associated with a student's disability that, you know, are entirely innocent. 

Lauren Daming: Beyond the potential for discrimination, we've seen, you know, some litigation on these issues. And some of the big litigation has been under the Illinois Biometric Information Privacy Act. That is a state statute that regulates how entities can collect and store and share biometric information about individuals. And in these cases, you know, the plaintiffs are alleging that this video monitoring is capturing images of their facial geometry and analyzing that and that constitutes biometric information and that the universities or the software companies are not following the requirements of that statute. 

Lauren Daming: Another legal issue we're seeing here are claims that this type of software and these companies are engaging in unfair and deceptive trade practices. There's been some attorney general complaints on this issue against many of the big companies. And the claims there kind of involve just a lack of transparency that these companies provide about how this software works, the type of data that's collected, how it's stored, who can look at it and how does the software even identify these red flag behaviors that, you know, they're turning over to professors and saying, this student is possibly cheating in this instance. 

Dave Bittner: I suppose on the one hand, you understand the need from the schools and from the teachers to try to prevent students from cheating, rather. And I think that's a legitimate challenge there. Are there any systems or organizations that are doing this in a way that's recognized as being better, as not having, you know, these sorts of issues or is this sort of a tension that's built into this sort of system, this sort of thing? 

Lauren Daming: Yeah. I think this challenge will always exist because, as you said, there are real benefits to this and schools need to preserve their academic integrity of their institutions. But any time you've got technology or video monitoring that's going into a student's home, there's going to be, you know, individual privacy interests that are at stake there. So I don't know if you can totally eliminate the concerns. 

Lauren Daming: But, you know, I do think since these concerns have been raised - and really students have been the most vocal with their open letters and petitions and complaints on Twitter. Since that's come to the forefront, I think the companies themselves have really taken notice and have been doing a lot of the work themselves to have external audits of their software, to, you know, assess potential biases, to see if they can improve it. You'll also see a lot of them have signed on to these kind of student bill of rights that talk about the rights that students have to access this information or to know the type of data that's being collected about them. And I think these companies have provided a lot more information about their own privacy policies and security practices and their attempts to comply with all the different federal and state statutes that are at stake here. 

Dave Bittner: Yeah. I can imagine, you know, as a student, if I am taking a test and, you know, I want to, I don't know, gaze off in the distance or stare at the ceiling as I try to think about a problem, and if I have a system like this, you know, triggering an alarm, saying, hey, you know, eyes on your paper, that sort of thing, I mean, that could get in the way of my ability to effectively take the exam. 

Lauren Daming: Yeah, of course. There's tons of anecdotal stories about students who have had bad experiences, who have cried during exams or, you know, become very stressed out and anxious about it. I actually have some personal experience with this. I took a - you know, strangely enough, I took a credentialing exam for a privacy certification earlier this year. And I took it... 

Dave Bittner: How ironic. 

Lauren Daming: Right, yeah (laughter). 

Dave Bittner: (Laughter). 

Lauren Daming: Yeah. I took it at home on my laptop due to the pandemic. And there was a live monitoring of that exam. I had to show photos of the room I was in. I couldn't let anybody come in. I couldn't go to the bathroom. And in fact, while I was taking the exam, I was looking away from my computer, and a little message popped up from the proctor that said, I noticed you're looking away from the computer. You need to stop. It was a little jarring to see that. So I, you know, completely understand the student perspective here. 

Dave Bittner: I have to say also that just, you know, personally, coming from the generation who grew up with our teachers telling us that, you know, we weren't always going to have calculators with us all the time - and, of course, now we're all toting around supercomputers in our pockets that have access to all of the world's information. I can't help wondering if this could be a catalyst for a different approach for how we come at exams, you know, that maybe this is an indicator that our exams aren't aligning with the reality of a real-world, professional position after you get out of school. 

Lauren Daming: Right. That is a great point. Some of the advice that we would give people looking to use this type of software is, because of the pushback, you need to consider alternatives to these types of exams. So that might be open-book exams, as you said, except more - you know, reflects reality more closely than a closed exam where someone is watching you. Yeah. So we - I think we might see it change kind of the landscape of how exams are given and how we do that part of education. 

Dave Bittner: You know, Illinois has, as you mentioned, sort of led the way with their Biometric Privacy Act. Are there any other things on your radar in terms of policy proposals that would address this specific issue? 

Lauren Daming: You know, I'm not familiar with any statutes that are in the works that are looking at this. I know that one thing that does touch on this is that many states have started to include biometric information in their definitions of PII in their data breach statutes. So even if particular states, you know, don't have any statute addressing this, there is some measure of protection for students against data breaches of this information. I guess the other thing kind of on the horizon - again, it's Illinois leading the way again. They have a particular statute in place that regulates how companies can use AI in their hiring practices. So if they're requiring applicants to submit video interviews as part of the hiring process, there's a statute in place that, you know, requires employers to get consent and provide some information about how the AI is used in that context. So, you know, this is an issue that goes beyond education. It's in hiring. It's in professional exams and certifications. And we're probably going to see a lot more of it. And so, you know, I won't be surprised to see more regulation or more concerns in the future. 

Dave Bittner: All right. Ben, what do you think? 

Ben Yelin: What a cool conversation. It's such a unique topic. I don't think it's something we've ever discussed on this podcast. And it's just increasingly relevant. My students in the past year have had to have these online, proctored examinations. You know, I think the goals of teachers, professors and academic institutions are certainly noble. They don't want their students to cheat. I get that. 

Dave Bittner: Right. Right. 

Ben Yelin: And for years, we had a system to protect against that. There are proctors in the classroom making sure that students aren't, you know, looking over their peer's shoulders and copying their answers. 

Dave Bittner: (Laughter) Right. 

Ben Yelin: So they've tried to apply that concept to virtual education. And for all of the reasons mentioned in that interview, there are a lot of really difficult complications, especially when we're using artificial intelligence. So, you know, it's not fair to students with disabilities if there's a suspicious movement and that's flagged to a teacher or to a professor when it's something that's part of that student's disability. I mean, I think that's really concerning. We know that these artificial systems are biased in all sorts of ways. So I think that's forced teachers and professors to, perhaps, rethink the, you know, examinations, whether to protect against these types of biases and to make the process more seamless, when possible, having open-book exams. 

Dave Bittner: Yeah. 

Ben Yelin: I know I've made my exams open-network and open-book over the past year because, you know, I'd rather not deal with the minutia of, you know, talking to online proctors and figuring out whether a student was actually cheating. And so I think that's something that a lot of teachers and professors are going to be wrestling with. 

Dave Bittner: How does that work out for you? I mean, as a professor, is it pretty clear for you when someone is, you know, just doing copy-and-paste internet research versus synthesizing their own answers? 

Ben Yelin: Yes. 

(LAUGHTER) 

Ben Yelin: So for any listeners who I've had as students, they know that I'm quite good at spotting plagiarism... 

Dave Bittner: OK. 

Ben Yelin: ...Especially when you copy and paste something not in your writing and it shows up in a different font. 

Dave Bittner: (Laughter). 

Ben Yelin: Copy and paste that into my own Google search. 

Dave Bittner: Got to put a little bit of effort in, right? 

Ben Yelin: Yeah, you've got to at least be creative there. I mean... 

Dave Bittner: Right. 

Ben Yelin: And that's not something that you're really even going to catch through an online proctor. That's why I sort of have questions about the enterprise entirely. I mean, I can understand - you know, you want to see if students are - they have cameras looking around the room to make sure that you're not - you don't have your phone next to you looking up answers or your textbook or something like that. 

Dave Bittner: Right. 

Ben Yelin: But so much cheating now could be done outside of the eyes of any camera if it is like a copying and pasting thing. And there are ways to protect against that. 

Dave Bittner: Yeah. 

Ben Yelin: I'm not sure that online proctoring is - the way that she discusses it - is really an answer to that question. 

Dave Bittner: Yeah. Yeah. All right. Well, our thanks to Lauren Daming for joining us - really interesting conversation. We appreciate her taking the time. 

Dave Bittner: That is our show. We want to thank all of you for listening. The Caveat podcast is proudly produced in Maryland at the startup studios of DataTribe, where they're co-building the next generation of cybersecurity teams and technologies. Our senior producer is Jennifer Eiben. Our executive editor is Peter Kilpe. I'm Dave Bittner. 

Ben Yelin: And I'm Ben Yelin. 

Dave Bittner: Thanks for listening.