Can You Really Tell if Something is Written by an AI?
Perry Carpenter: Hi. I'm Perry Carpenter, and you're listening to 8th Layer Insights. It's 3am. Your phone buzzes with an urgent message from your best friend. They're stranded in a foreign country, desperate for help. Your heart braces as you read their plea. But wait. Is it really them, or is it just some AI-powered scam bot playing on your emotions? Now flash to a college classroom. A professor beams with pride reading her brilliant student's analysis. It's insightful, eloquent, and perhaps the best work they've seen in years. But a nagging doubt creeps in. Is this the product of a young mind, or is the mind behind this paper artificial?
Speaker 1: Breaking news.
Perry Carpenter: Now the breaking news chyron flashes across the screen. A world leader makes a shocking announcement that could spark international conflict. But, in this age of deep fakes and synthetic media, can we trust our own eyes and ears? Can we really trust anything? Welcome to 2024, where the line between human and machine-generated content isn't just blurred. It's practically invisible. But here's where it gets tricky. We are not just dealing with a black and white world of human versus AI. This is a spectrum, a messy gradient of human and AI collaboration. I mean, think about it. How many of us use spellcheck or grammar tools or predictive text every day? We are all cyborgs now, augmenting our writing with AI assistance. Some of us might use AI to brainstorm ideas or to polish prose. Others might just let it take control, generating entire articles or reports. And let's not forget about the bad actors. They're not just using AI. They are weaponizing it, creating armies of bots to spread disinformation, to craft personalized scams and even impersonate our dearest family and friends. So where do we draw the line? What is authentic human expression? And what's synthetic? Is a human-written article polished by AI tools more quote, unquote real than an AI-generated piece that was prompted, edited, and shaped by a human? When does augmentation become automation? Solving this puzzle isn't just hard. It's a constantly moving target. As we develop better detection technologies and methodologies, AI models are becoming more and more sophisticated. This is an arms race where the finish line keeps moving towards and past the horizon. And the stakes, yeah. They couldn't be higher. And we're talking about the future of trust, of human creativity, how we communicate and connect to the people in the world around us. But don't despair just yet. There are brilliant minds working around the clock to tackle this challenge, digital sleuths developing cutting-edge techniques to unmask synthetic content. But, as I said earlier, this is a moving target. And the arms race isn't just related to the technology at play. It's also the mindset and philosophy around what is authentic, what is synthetic, and what kind of poly cotton blended reality we decide we're okay with. My guest today is Jon Gillham. He is the founder of a detection platform named Originality.ai. Jon's career and company is dedicated to working on the problem of detecting synthetic content, specifically, content that has been written or largely influenced by AI. And so, on today's show, what makes something real in 2024? How do we know, and why does it matter? Welcome to 8th Layer Insights. This podcast is a multidisciplinary exploration into the complexities of human nature and how those complexities impact everything from why we think the things that we think to why we do the things that we do and how we can all make better decisions every day. This is 8th Layer Insights, Season 5, Episode 7. I'm Perry Carpenter. Welcome back. Let's dive right into our interview with Jon Gillham.
Jon Gillham: My name is Jon Gillham. I'm the founder and CEO of Originality.ai, which is a content marketing platform that helps users receive piece of text and then make sure that it meets their editorial standards. And so it's most commonly used for understanding if a piece of work was written by a human or written by AI.
Perry Carpenter: And that's, I think, what I want to dive into. So, for you, when you say this is for somebody to decide if a piece of text is written by AI, what is the outcome of that for the person that learns it? Is it just general awareness? Is it that they should not trust it? Is it that it's not accurate? I think there's all these different perceptions that people have when they run these things through different tools. But, at the end of the day, what are you hoping that people get?
Jon Gillham: Yes. It depends on the user. Our primary user is somebody that's getting content from a writer that they've hired and are going to be publishing it on the web. That's our primary user. So I can answer for that. For them, what I'm hoping that they get out of the tool is an understanding on if that piece of content meets their specifications that they want to publish on their site. Publishing AI content, a lot of people believe produces -- and certainly I do, introduces risk to your website in the form of Google updates. And so, by knowing if that content was AI generated or human generated, you are the one that is making the risk-based decision on whether or not you should publish that content on your site.
Perry Carpenter: Right.
Jon Gillham: So that's kind of like the most important part. The other part is just general fairness. If you're paying a writer $100 or $1,000 for a piece of content and they just copied and pasted it out of ChatGPT, you're getting a pretty raw deal.
Perry Carpenter: Yeah. And I -- I totally agree with that. I think, though, I mean, if we're -- if we're to look a couple years into the future and think about the fact that AI will probably end up touching every piece of writing at some point, I think most of the tools on the market say, if you -- if you're even using Microsoft Word at this point and it's given grammar and punctuation, spelling suggestions and all that, that there are some fingerprints of AI on a piece of writing. So where do you think the crossover point between trusting a piece of content and not trusting it or seeing value in the thought behind the piece of content and not seeing that value starts to come in?
Jon Gillham: Yeah. So I think it's a great question. So we're not anti-AI. Like, we use it all the time. We -- I use it. I would prefer to sort of communicate in spreadsheet format all the time. As an engineer, I like spreadsheets more than I like words, right? And so I use it all the time to sort of, like, translate my thoughts into the writing. And so I think if you know the author and, you know, where's this -- where's this go, like, have projected those two years when AI is touching a lot of parts, you want to know, I think, the authorship behind a piece of work, whether that's your podcast, whether that's an email that I send that with my name on it. I think that's the method that people will have the confidence, and it won't be as much about whether it's AI or human generated.
Perry Carpenter: Right.
Jon Gillham: I think, for the end user, that's true. I think where that can break down is in the face of the amount of AI spam that some algorithms that we are needing to deal with, like, for example, Google --
Perry Carpenter: Yeah.
Jon Gillham: -- being overrun by AI, spam is an existential threat for them. And they need to fight that. And the evidence that we're seeing is that they are fighting that by going after AI spam. And if you are publishing AI content, you are increasing your chance for being included in collateral damage.
Perry Carpenter: Yeah. So, in a second, I want to circle back to Google because I think that feeds straight into detection tools. But, before we do that, why don't we think about a couple other ways that AI generated texts can go wrong, right? So one of those would be, as soon as you can generate AI content at scale, all of a sudden you've got bots and trolls and everything else that can function at levels that they never have before. So this contributes to disinformation and misinformation and kind of the state of social media and information warfare. Where do you think tools like this fit into that?
Jon Gillham: So tools like this being -- actually tools like this being the AI itself.
Perry Carpenter: I was primarily thinking detection, but I'm really interested, since you spend all day thinking about all these problems, I'm interested in any perspective you have.
Jon Gillham: Yeah. So I think these tools, the AI generation, LLMs and generative AI makes the creation of content, the creation of words incredibly efficient. You know, obvious statement. And it results in an efficiency lift for anyone that is in the business of publishing words. And so, when that comes to misinformation, that comes to trolls, its headquarters made do they are capable of publishing machine content. Platforms face this, child platforms that provide eyeballs on user-generated information are faced with an incredible challenge of needing to be able to draw a nuanced line between what is acceptable and not acceptable. And in the face of generative AI, Medium has taken the stance around we're about human stories, not AI-generated stories. And all companies sort of like going down the line need to take a stance around -- around these tools. Reddit has an Army of moderators who are great at -- they have decades of sussing out and expelling bad actors. I think they're going to -- they're doing really well at dealing with AI content. Review platforms have had less history with needing to deal with this level of mass-generated content and are struggling. And so I think there's going to be this whole range of how they -- what controls they put in place. And I don't think it's -- I think protection is a piece of that. But I think there's a lot of other signals to spam and misinformation and document and text that harms humanity as opposed to have benefits within their platforms. I think detection is a piece of it but not the only solution.
Perry Carpenter: So one of the things that you mentioned is, when you're getting into things like is this original, will even Originality.ai, your company, you know, the thing is in the name, right? You're trying to see what is the origin of the piece of text or the piece of content that's there? And one of the primary use cases you mentioned was avoiding penalization or bias from Google if it perceives that AI is part of that. OpenAI has put out a report saying that they don't really see value. And Mozilla Foundation put out a report recently. It says that they don't see value in them. So you do have really, really smart, capable engineers on both sides of the conversation. Google seems to believe differently. You seem to believe differently. So break down where the differences in opinion are and maybe some of how the technology works.
Jon Gillham: So, yeah. So I can speak to -- speak to it. Hopefully in -- yeah. We've explored this topic in extreme levels of detail. So I think the most important thing to know is that false positives and false negatives do happen. Detection tools are not perfect. They are classifiers. They get things right, and they get things wrong. Sort of simply you can think about it a little bit like the weatherman. There's a forecast for rain. It might rain; it might not rain but doesn't mean that you're going to totally disregard it the next time.
Perry Carpenter: Right.
Jon Gillham: So there's some chance that it works, some chance it doesn't. And then the question on whether or not they're effective is a question on what level of accuracy do you need for your specific use case?
Perry Carpenter: Right.
Jon Gillham: Within academia, we advise against our tool being used within academia because it needs to achieve a academic -- academic discipline level of enforceability. And detection alone does not do that. And so that's -- that's where it doesn't work. Does AI detection work? Not when you need 100% certainty. Does AI detection work if you can live with 98% accuracy, 2% false positive rate and its standard web content. Absolutely. We've tested against every dataset that has become available. We have created an open source tool to let others run their own dataset through multiple detectors. Put in your own API key; it runs the dataset against our detector and others and then produces the statistical analysis to just communicate the efficacy of the classifiers against your body of work. And so we've done everything -- we're trying to do everything we can to continue to educate on the limitations of these tools, which there are some. But that doesn't mean that they're BS. And we have a kind of looping back to OpenAI, we have a bet out there that we're hoping they will take us up on but -- or anyone else that thinks that AI detectors don't work that will create a new dataset. They can create the dataset. Every one we get right, they donate to charity. Every one we get wrong, we'll donate to charity. No one's taking us up on that one yet so still waiting.
Perry Carpenter: Well, maybe you can poke Mozilla into it now that they're going to make their statement public.
Jon Gillham: Yeah. I'll say OpenAI was faced with a really challenging situation. And so I understand why they shut their classifier down.
Perry Carpenter: Yeah.
Jon Gillham: They were viewed as the authority on, if their detector called something AI, even though they would still get it wrong sometimes because there was a classifier and it has false positives, it was causing a lot of pain because it was being viewed as it must be accurate because this is the maker of the -- of OpenAI. And so, as a result of that very, very, very strong bias that they would have internally to make sure that false positives were near zero, it still wasn't zero. And then anytime you're sort of making this trade-off between false positives and -- and false negatives, so, like, how accurate is it calling AI, AI versus how accurate is it calling humans human. Always a little bit of a trade-off there when you're building classifiers. And they would have had theirs tuned so far to the side of human to reduce false positives that their accuracy rate was absolutely garbage. And so they had a low but not zero false positive rate and then a pretty useless detector at identifying AI content.
Perry Carpenter: Okay. In a second, I think I want to get into, like, what are some of the algorithms or what are some of the touch points that these tools would look at to make that determination? But maybe let's start from the other side. If AI has a decent chance of -- I think you said 98 to 2 in some circumstances. If it has that kind of accuracy in being able to detect whether something is written by AI or a human and have a confidence level, why can we not get the generic AI output to then replicate something that looks human? I mean, it knows what human looks like. Why does it not do that? Is it just an artifact of next-word prediction or token prediction, I should say?
Jon Gillham: Yeah. So this is kind of one of the fascinating things about this project from my -- from my end is that we don't know what it sees.
Perry Carpenter: Yeah.
Jon Gillham: Yeah. It's one of these kind of crazy, like, what is it identifying as AI? We don't -- we don't know. And it's somewhat unsettling of an answer. It's unsettling to us. It's unsettling to you. But the truth is that humans are uniquely good at thinking we see patterns but actually really bad at it where we can create patterns out of noise that just don't exist, you know, this gambling and thought marker gambling that is sort of built off of that, that ability when humans think that we see patterns, AI is truly good at identifying patterns that we can't -- we can't recognize. And so what are the -- what are the commonalities about content like detection of AI versus it doesn't, and what is it identifying? Yeah. An unsettling answer, but it's a it's something that we don't -- even as the creators don't know.
Perry Carpenter: Yeah. I'm just wondering because I've actually not tried this yet. I've done a lot of work with prompt engineering to try to get AI content to where it's less detectable and with varying degrees of success, especially when I can feed in a corpus of work of stuff that I have written myself before and say try to emulate this. It gets way closer. But using something like multi-shot prompting or chain of thought, it seems like you'd be able to say, you know, create output X. Then review output X and give a confidence level on what you think it would be as far as being predictably AI or not. And then make adjustments that would make it not that. And, for some reason, it seems like even it's not able to do that because I've tried that kind of test across several different tools, and there's still markers of AI, even after multiple iterations.
Jon Gillham: Yeah. I can empathize with that frustration because we live that every day, as well, trying to beat our detector. So we have a red team and a blue team where we're always trying to beat ourselves and other detectors with any method that that is available, including many of the ones that you just listed --
Perry Carpenter: Yeah.
Jon Gillham: -- around loading a corpus and trying to -- try to emulate that writer and an iterative approach of recycling until -- until it passes. And so it can be -- and then trying to replicate whatever -- whatever, when we were able to get to trick --
Perry Carpenter: Right.
Jon Gillham: -- replicate that approach across more. And it can be -- it's really challenging to find methods that bypass consistently that produce quality work, that if you want to bypass detection, strip out all formatting, introduce spelling mistakes, and you're good. You can bypass it that way.
Perry Carpenter: Yeah. So one other interesting thing that came up with multiple tools is I did try to generate a piece of AI text. Well, I did. I mean, that's what they do. And then tried to say what could I do? What small changes could I do to make it look human? And I got it to move from 100% AI detection to 100% human detection. But in these tools similar to yours, and including yours where it color codes things, every sentence was color coded as like 90% or above probability for being AI. But then it showed 100% probability for being human. What goes on in those kind of circumstances? Because that's perplexing.
Jon Gillham: Yeah. That's -- I've already done it recently. We've helped fix that. So we always have a trade-off in the tool around the resolution of detection and the overall accuracy of the document. And so that's -- that's where that issue is coming from. It's a known issue where -- we've made, actually, a recent update, like a week ago that has significantly improved.
Perry Carpenter: Okay. It was about two weeks ago that I tried that. Yeah.
Jon Gillham: Yeah. So it's frustrating when that happens. And we understand why. So it's -- we want to provide -- ideally, we would provide a word level of like, hey, what part of this content was AI generated? And we want to provide as high a resolution as possible.
Perry Carpenter: Yeah.
Jon Gillham: As few words as identified, but then we'll want to provide an overall score that is as accurate as possible. And the more words that these captures can evaluate on whether or not the content was AI generated or human generated, the more accurate it becomes. So sort of the accuracy rate is really bad below 50 words, like --
Perry Carpenter: Yeah.
Jon Gillham: -- flip of a coin terrible. And then 50 to 100, it gets pretty decent. And then beyond 100 words we're getting diminishing, diminishing returns. So we want, the longer the words, the more accurate our overall score will be. And -- but the result is that we've -- when we have our highlighting model that tries to provide as high resolution as possible, and those will not always be aligned. And that can understandably cause confusion where you're getting an overall score that says one thing, but then if you look at all the highlighting, it says something different. So that's probably what you experienced. We've pushed an update that hopefully addressed that or significantly improved it, but it's still -- it's still a fundamental issue on the trade-off between resolution and accuracy.
Perry Carpenter: Right. And then, when it's your tool or any of the other tools out there and you're given that percentage, how should somebody read that and interpret it? And, like, what should it mean to them when it says, you know, let's say it's 80% AI detection, 20% human? Would that mean that the person on the other side, when they're evaluating that, should say 80% of the content has been generated by AI and 20% human? Or is it a confidence level? Or is it just that 80% has been somehow touched by AI?
Jon Gillham: Yeah. No. Great, great question and one we're working on helping to communicate more accurately. It's saying the probability that AI was involved in the creation of that document. So the probability that that document was meaningfully edited and impacted and changed by AI. So it does not mean 80% AI; it means that we are at -- the detector 80% confident that AI created that document, not that 80% of it was AI and 20% was human. Yeah. Great, great question. I think probably one of the most common misunderstandings with these tools.
Perry Carpenter: Yeah. For published authors who are writing books and everything else, there's usually three or four rounds of editing involved with that. So by the time something comes out, it's pretty close to good. Still, I see typos in books all the time. So they're definitely not all created by people. But those multiple rounds of editing, if somebody were to take a finished work of an author that has been really scrubbed by multiple rounds of editing, would that have a possibility of showing up as being AI generated as a -- as, like, a false flag?
Jon Gillham: We haven't seen that. So when we -- I'll probably talk more about this when we talk about how it works under the hood.
Perry Carpenter: Yeah.
Jon Gillham: But the train sets that we build our tool around are -- is typically web content gets you to cross many use cases, including publishing. We've seen a drop in accuracy based on the time period at which it was written but not on modern published works that we -- datasets that we've run through have not had a drop in accuracy relative to our sort of standards that we communicate. Historical works can just because the format -- the formatting is different. The writing style is different. And we're -- and we're not trained. That's not what our users are using it for.
Perry Carpenter: After the break, the conclusion of our interview with Jon Gillham. Welcome back. Let's talk a little bit about how the tech works because my assumption as I went into this was that it was almost like a probability model based on next token prediction, and you were kind of starting with a piece of content and then saying, what's the probability that, after this set of words, the next word would be x in the same way that an LLM would work. Seems much more detailed and complex than that rudimentary way that I would have first assumed that it would have worked.
Jon Gillham: Yes. So the way that you assume, there's a great -- I'll send you the link. It's a GLTR. It's an open source solution that provides that way of sort of a linear probabilistic approach to saying how surprised am I that this word shows up. Not surprised at all. That that made sense to be the next word. And then summing up that level of surprise and then giving a -- giving an overall score. So that -- that was what some of the earliest detectors were. That was effective on GPT-2. No longer effective on GPT-3 because you were able to prompt shot GPT or GPT-3 and then shoot GPT to say write like so and so. And then your probability of the next word will no longer became -- without the understanding of the prompt, your probability of the next word became very, very different.
Perry Carpenter: That makes sense.
Jon Gillham: So if it said, write like Perry Carpenter versus write like Shakespeare, the probability of the next word became very different and not native to the model. So that was a model -- that was a method of detection that was effective on GP-02 before you could send it in so many different directions like GPT-3. And then, really, that's a GPT-4 property. That's pretty incredible. So that's how they used to work. There's also some I call them bag of words attempts at detectors where it's like how many times does this word show up? So, like, there's methods of saying, well, there's higher perplexity or lower perplexity or burstiness, or the readability score is in -- and there's some truth to some of these where the readability score of LLM-produced content is tightly and normally distributed around 11th grade or 10th grade, whatever, whatever it might be. However, again, you're very easy to say to the LLM, say, write with a grade 12 writing readability. Write with a grade 9, 8, 7, whatever you choose. And that would make the sort of bag of word detectors ineffective. And then there's a sort of unsettling third approach, which are -- which ours is, which is a supervised learning approach. Feed it millions of records or hundreds of thousands of records of AI content and human content. If you picture a Venn diagram, you try and make it as tightly overlapped with the type of content that your users will want to use them and the detector's built for and then have it -- have the AI do what it is exceptionally good at and learn to tell the patterns and the differences between the two.
Perry Carpenter: With these detectors, are there any learning structures like GANs or anything else in place where you're pitting AI against AI to self-learn?
Jon Gillham: So not in the models. In the dataset creation, yes.
Perry Carpenter: Okay. Interesting. What do you think would be the most common misunderstanding of detection technologies as specifically LLM detection that people who are unfamiliar or moderately familiar with LLMs might have coming at somebody maybe at my level or less educated?
Jon Gillham: Yes. I think there's two, and we touched on the one. But we've actually touched on both. But there's two common misconceptions, one that they take a binary view of what detectors are either perfect or detectors are BS, and neither are true. Detectors are effective within certain parameters. For our use case, web content, 99% accurate identifying AI-generated content. One and a half to 3 1/2 percent false positive rate at miscalling human-generated content AI-generated content. That's misconception number one is they think either they're BS or, no, they're perfect. And then misconception two is that the score -- and we touched on this, but the score, 75% is human, she made 25% AI means that the tool is 75% confident that that was a human-generated piece of content and not that 75% was human, 25% was AI. Although it's frustrating, you want to see it identify your 100% written work as 100% original, it's still correctly identifying that piece of content as human-generated content.
Perry Carpenter: Okay. I guess one other question related to maybe the tech under the hood, would something like going into one of the playground environments for like ChatGBTs or OpenAIs, I guess, to be more exact, and futzing around with the temperature a little bit. So rather than using the default temperature that's in the tool so you're increasing chaos a bit, would that affect the way that these tools work?
Jon Gillham: So used to, yes. I think that was one of the sort of earliest methods of bypassing, similar to other strategies that have been around to bypass detection. We build a dataset so we -- we identify a dataset against it and then train on it. So right now that answer would be, if you push it to the point where it produced gibberish, it will reduce the efficacy of the section.
Perry Carpenter: Right.
Jon Gillham: But we sort of have built datasets and I think other tools that would, like, be done similar, build datasets on settings that will produce useful content and then train on them to ensure that you can stay back to similar. One of the other sort of common methods of attempting to bypass detection is to use one LLM to write and then another modeled to paraphrase and then using that to try and bypass. That also is -- is, yeah, no longer effective.
Perry Carpenter: Ah. That's great. So we talked about common misconceptions. If you were to think about the state of now, what is the one thing you wish could improve the most rapidly when it comes to AI detection? That could be anything from the technology to public perception to use case application and so on.
Jon Gillham: The genie's out of the bottle with AI content creation. It's causing real pain to writers. And when we have a false positive, we add to that pain. I wish we -- if I could wave a magic wand and achieve anything would be 0% false positive rate. A false accuracy rate of sort of like an accuracy rate of 99% on AI content, that's great. False positives do cause pain, cause pain in academia, cause pain with writers. We'd love that rate, that to be at zero. I think we are producing tools that help deal with false positives when they do happen, a free Chrome extension to help people visualize the creation of a Google document so, if they do get a false positive, they can then share the creation process of a document to show that that was truly created by me.
Perry Carpenter: That's great.
Jon Gillham: So I think that's what, if we could achieve anything, that that would be -- that would be what we would want to achieve.
Perry Carpenter: Yeah. I think that's great. You mentioned academia a couple of times, and I think that triggered a memory in my head about one of the reports that I read, which was one of the things that tends to create false positives more is academic writing or English as a second language, especially in academic or engineering context. What's -- what's going on there? Is that the rigid structure? Is it word choice and flow? Or -- or is it some weird thing that I've not even thought of?
Jon Gillham: Yeah. So we don't train on academic datasets for our current models. We do have a model that we're working on for that. We don't love to use case in academia. That's not who we're building our tool for. And so our model, we've tested against still many academic datasets and performed well, not as well as web publishing content. So I think that's part of it. We do terrible on legal text, patent documents. Again, not our use case.
Perry Carpenter: Right.
Jon Gillham: And the auto formatting is and the less data rich that formatting is in our training dataset, the less accurate the tool is. And so we try and -- we try and really define where our tools should be used so hopefully it doesn't get misused in environments where the accuracy rate drops off. And, then again, tried to provide as much open source tooling to help evaluate efficacy. The English as a second language narrative is -- has been sort of shocking to see how it has proliferated from one really bad study. And when I say bad study, I say 91 a 91 sample study to evaluate AI detector efficacy is not a very good sample size. It was just those 91 samples were taken from a student forum with no control for AI creation from 2019 to 2023. So they sort of ignored AI creation pre-ChatGPT. They compared a nonnative English grade 12 writing to grade 8 native English writing. So you have not a fair comparison. And they used LLMs to fine-tune human writing and then still tried to call that human writing, which we correctly identified as AI-generated writing because it was AI-edited and significantly changed documents. So we collected other datasets, IELTS, so a nonnative English entrance essay, a larger -- still not large enough but a significantly larger sample size of 1500. And in that study, again, this is a academic writing less in line with our tool's use case. And we had a 95% accuracy rate on detecting a human as human and a 5% false positive rate, a wildly different than their advertised 30% error rate. So we're -- it's been fascinating to watch this narrative around detectors are bias against nonnative English writers proliferate from a single study that had -- and sample size of 91 really taken samples.
Perry Carpenter: You know, that's the -- that's the danger with any study, right? We see this over and over and over again, that one poorly designed study or slanted study can impact perception for years because the headline is shocking. Journalists and well-intentioned people everywhere pick it up and distribute it. Sometimes they reference back. Oftentimes they reference five or six other articles that reference that, and then so it starts to build an authority base; and nobody remembers where it actually came from. So you get in like footnotes, what I just call footnote pasta, kind of like copy pasta on -- on Reddit. And it creates this received wisdom or received truth that then is really, really hard to correct because we see over and over and over again that the false narrative that is shocking, especially when you're saying bias against a people group, will travel and be retained by our memory like seven to 15 times farther, faster, and deeper than the true narrative. And the retraction is not near as shocking, and so it doesn't have that emotional impact or resonance and doesn't get replicated like that. >> Jon Gillham:. Yeah. No. I think -- I mean, it sounds like this has been your well and something that you've -- you've thought about deeply. But this has been -- I think I had held academia in a higher -- a higher, higher authority than then I have from sort of some of this work where I've seen just this was the -- they had to have known. I expected significantly better. And then to have seen how that one paper has been just for, like, exactly as you described, people referencing studies, studies which references multiple articles that are all linking back to the same one flawed study. Yep. Yeah. That I think that's not just indicative of academia. I think it's indicative of any narrative or bias or shocking truth that we come across. And we -- you know, we're in an era now where everything is about information proliferation and filling information vacuums and creating narratives that are shocking. And, when you have that kind of situation that we're in as a society globally, it's, you know, a hotbed for that more now than ever. And so I think a lot of the fear, and maybe the uses that AI detection will really work well in is countering some of that as we go long-term and trying to stop or at least flag the dissemination of that at its root and as it's happening so that people are at least aware when this is being distributed in that way. So interesting times.
Jon Gillham: Yeah. The societal cost of being unable to truly identify what was -- was -- has a human behind it and what was just AI-generated I think is -- is not zero. It's significant and in ways that we don't fully understand online reviews being untrustworthy is sort of an obvious one that we're first seeing Google results being -- declining as a result of AI spam overrunning them, actually, I think is another one that we're living through right now.
Perry Carpenter: Yeah. All right. So three fun questions that I usually ask, especially security people. But I think they're fun for anybody. Actually, we can narrow it down to two just so you don't get on -- put on the spot for security stuff. And we already answered the third one that I usually ask, which is about myths and misconceptions regarding the thing that we're studying. Number one, for all of us, we have weird interests whenever we're doing research and leaves a little side trails that we have to go into. So given that, if somebody were to look at an entry in your browser history and take it out of context, which would be the hardest to explain?
Jon Gillham: Probably stuff I'm doing right now with my -- with my son, helping him build some Roblox calculators.
Perry Carpenter: Okay.
Jon Gillham: And that would probably be some -- the most -- someone would think I was really obsessed with the Roblox economy.
Perry Carpenter: Cool. So the economy itself, so the -- talk about that for a second.
Jon Gillham: Yeah. So in the game of Roblox, there's robux. And then you can turn those -- you build things in the game that they then get turned into robux. And then you get a piece of them. You can sell them to other users who have bought robux for dollars. And then Roblox, the company, takes a -- takes a cut and takes a tax and understanding how that sort of economy has been designed to function is fascinating.
Perry Carpenter: Interesting. Yeah. I can imagine. All right. And then related to your field of study or not, what is one book that you think this audience should be aware of or read? And it doesn't have to be a book. It could be a podcast or a documentary or movie, anything else.
Jon Gillham: I think it's -- I think it's a rare -- so to call it generative AI is a rare field, where many of the leaders in that -- in the companies that are in this field are doing more sharing than I have seen in any other time. So Sam Altman, Ilya, Schezwan, Andrew Yang, like, there's just so many that are sharing so much that I think it's a unique time to be living through a technological revolution, while those that are at the forefront of that revolution are sharing so much about how they're thinking about it. I think it's -- they're such incredible speakers and so incredibly smart that it can be easy to believe 100% of what they're saying without listening critically to it. And, knowing that, they obviously have a bias, the same as anyone else that that lives has a bias. But I think that's in terms of what -- what I think is unique right now is I think the -- the speed at which this is moving generative AI progress and the openness of so many of the people that are driving that progress to be sharing so probably is really unique and I think a bit -- bit of a -- bit of a gift right now for us trying to learn and understand and what everything is that is happening.
Perry Carpenter: Yeah. It's clear that AI's influence on writing isn't a passing trend. It's becoming deeply integrated into our creative process. From basic spell checkers to advanced writing assistance, AI is weaving itself into the fabric of how we communicate. The distinction between human- and AI-generated content will continue to blur, presenting both challenges and opportunities. And so what does this mean for us as we navigate this evolving digital landscape? It calls for a delicate balance, cultivating healthy skepticism without succumbing to paranoia. It invites us to cherish the nature of human creativity while also embracing AI's potential as a tool. It challenges us to be more discerning than ever about our information sources. So, as we forge ahead, let's commit to using AI responsibly. Let's sharpen our critical thinking skills and maintain that essential human touch in our digital interactions. After all, it's our uniquely human traits, our experiences, emotions, and perspectives that truly give life to our words. In this world increasingly shaped by artificial intelligence, our natural intelligence, our capacity to question, reason, and empathize becomes more precious than ever. And oh, yeah. This last section, that was written by AI. So, yeah. And, with that, thanks so much for listening. And thank you to my guest, Jon Gillham. I've loaded up the show notes with more information about Jon, some great posts from his blog, and a few interesting bits of research that relate to the topic of detection methods and efficacy. If you haven't yet, please go ahead and subscribe or follow wherever you'd like to get your podcasts. And I'd also love it if you'd tell someone else about the show. That does really help us grow. If you want to connect with me, feel free to do so. You can find my contact information at the very bottom of the show notes for this episode. The 8th Layer Insights branding was created by Chris Machowski at RansomWear.net -- that's W-E-A-R; and Mia Rune at MiaRune.com. The 8th Layer Insights theme song was composed and performed by Marcos Moscat. Until next time, I'm Perry Carpenter, signing off.
