Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

The Deepdive

The Deepdive artwork

OpenAI Watermarks: The Signature Forgery Problem

Oct 7, 202620 minAudio
ShareXLinkedInFacebookWhatsAppEmail

A viral cartoon hits your feed: Dolly Parton, Tim Curry, the Pearly Gates, and a New Yorker cartoonist’s signature tucked neatly in the corner. It’s funny, polished, and instantly shareable. There’s just one problem: the artist whose name is on it never drew it. That single detail opens a trapdoor into the most unsettling part of generative AI, where “authentic-looking” can be manufactured at scale while real human identity gets treated like a reusable asset. We break down OpenAI’s invisible text watermarking system, Text Grain, and why it’s engineered like a cryptographic provenance machine. You’ll hear how the “loaded dice” approach embeds a fingerprint into word choice, how the detector is kept behind a velvet rope, and how a little rewriting with synonyms can cause AI-generated text detection to collapse. On paper it’s rigorous. In the wild, it’s brittle, gated, and easy to evade. Then we pivot to the image side, where the stakes become personal. AI image generators aren’t just borrowing an art style, they’re outputting cartoons with living artists’ handwritten signatures, down to tiny habitual marks. We connect that to the training-data pipeline, unsealed court records, pirated datasets like LibGen, and the legal catch-22 that makes non-commercial viral fakes hard to prosecute under today’s copyright law and right of publicity rules. If you care about AI authenticity, artist rights, copyright, and what “proof” will even mean online, this conversation is for you. Subscribe, share this with a friend, and leave a review with your take: what should count as real in the age of perfect imitation? Leave your thoughts in the comments and subscribe for more tech updates and reviews.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

AllanImagine it's um late August 2026. You're scrolling X and you stumble across this viral cartoon.

IdaOh, right, the Dolly Parton one.

AllanExactly. So it features Dolly Parton in this massive sparkling gown, and uh Tim Curry is there too, dressed in full Dr. Frankenfurter gear.

Speaker 1Iconic duo.

AllanRight. And they're arriving together at the Pearly Gates. Dolly looks at God, who's apparently working the reception desk, and she says, I knew I chose the right plus one. It's just a classic single panel gag.

Speaker 1It really is.

AllanIt's got like 25,000 likes. And down in the bottom right corner, you see the signature B Loper, which is the iconic handwritten pen name of New Yorker cartoonist Brendan Loper.

Speaker 1Yeah, it feels like a perfect internet moment, except for, well, one fairly massive detail. Which is Brendan Loper never drew that cartoon.

AllanNo way.

Speaker 1Yeah. He actually found out about it when his twin brother texted him a screenshot. A Dolly Parton fan had simply logged into Chat GPT, asked it to generate a New Yorker style cartoon, and the AI didn't just, you know, mimic the classic black and white art style. It perfectly, flawlessly forged Loper's exact handwritten signature.

AllanAaron Powell I mean, I love that this exists, but also why? Like, why would a machine even know how to do that?

Speaker 1Aaron Powell That is the big question.

AllanAaron Ross Powell And welcome, by the way, to you listening to this deep dive into what has to be the ultimate corporate paradox of October 2026.

Speaker 1Aaron Powell It's a jarring one, for sure.

AllanAaron Ross Powell You're really going to want to stick around for this because we are actively deciding what authenticity actually means right now. And it turns out the machines are currently giving themselves significantly better copyright protections than the human beings who trained them.

Speaker 1Aaron Powell Yeah. Watching this unfold in real time is wild because if you look at that exact same news cycle, literally the same week, OpenAI's image generator is caught committing digital identity theft against living artists. The company is over on their corporate blog doing a victory lap.

AllanAaron Powell Right, about their own text.

Speaker 1Aaron Ross Powell Exactly. They were proudly announcing this incredibly dense, mathematically perfect system to prove the authenticity of its own AI-generated text.

AllanAaron Ross Powell This obsessive need to protect their own data makes what their image models are doing out in the wild feel um incredibly hypocritical.

Speaker 1Aaron Powell Oh, absolutely. The contrast is staggering.

AllanAaron Ross Powell So before we get to the whole cartoon forgery ring, let's start with their state-of-the-art security system. What exactly did they build for their own text?

Speaker 1Aaron Powell Right. So they introduced an invisible text watermark called text grain.

AllanAaron Ross Powell Text Grain. Yeah. Okay.

Speaker 1Aaron Ross Powell This was built specifically for chat GPT and codex outputs, primarily to comply with the European Union's AI Act, which basically mandates that AI-generated text has to be machine identifiable.

AllanAaron Powell Makes sense.

Speaker 1But under the hood, the engineering is legitimately encyclopedic. Like the technical report outlines how the system partitions the AI's entire vocabulary into keyed blocks.

AllanOkay, I'm tracking.

Speaker 1And then it solves an entropy calibrated optimal transport problem using something called gumble random costs combined with coolback libeler regularization.

AllanOkay. Wait, hold on. We have to translate that jargon.

Speaker 1Fair enough, yeah.

AllanBecause entropy calibrated optimal transport sounds like, I don't know, a Star Trek warp drive malfunction. What is actually happening to the words?

unknownAaron Powell Okay.

Speaker 1Think of the AI's vocabulary as a giant dictionary.

AllanRight.

Speaker 1When ChatGPT writes a sentence, it's essentially predicting the next word based on probabilities. What TechSgrain does is secretly use a cryptographic key to subtly manipulate those probabilities.

AllanOkay. So it's like loaded dice.

Speaker 1Exactly.

AllanAaron Ross Powell The model still rolls to pick the next word, but OpenAI hid these tiny statistical weights inside the dice so that over, say, uh 400-word essay, a highly specific pattern of lucky rolls emerges.

Speaker 1Aaron Powell That's a perfect way to look at it. The optimal transport part of the math is just figuring out how to nudge the AI toward those specific words without, you know, ruining the actual meaning of the sentence.

AllanAaron Powell Right, so it doesn't sound like gibberish.

Speaker 1Yeah. And the callback librar regularization, that's the leash. It stops the AI from picking bizarre words just to satisfy the secret watermark. It ensures the writing still sounds totally natural. Aaron Powell Okay.

AllanI get the loaded dice thing, but doesn't that inherently make the writing worse? I mean, if you're forcing it to pick specific words to form a secret code, it's not picking the best words anymore.

Speaker 1Aaron Powell You would think so, right? But here is the wild part. OpenAI fiercely defends the quality of this system.

AllanAaron Powell Really?

Speaker 1Yeah, they ran extensive tests on their GPT-6 astro model, and the watermarked text scored a 93.94% on the GPQA diamond benchmark.

AllanAaron Powell Okay. And what was the normal score?

Speaker 1Aaron Powell For context, the untouched, unwatermarked text scored a 94.44%.

AllanWow.

Speaker 1It's virtually identical.

AllanAaron Powell That's insane. So they invented this microscopic statistical lock for every single word, baked right into the seams between synonyms, and it doesn't degrade the intelligence at all.

Speaker 1Aaron Powell Well, it actually gets weirder. Of course it does. On the terminal bench evaluation, the watermarked text actually scored higher.

AllanWait, higher?

Speaker 1Yeah. It got a 56.06%, while the unwatermarked version only scored 53.90%. Aaron Powell That makes no sense. I know. But essentially they built this flawless provenance tool that proves exactly where a text came from, completely invisibly.

AllanAaron Powell Okay, but here's the catch with this flawless system. It is guarded behind the strictest velvet rope imaginable.

Speaker 1Oh, heavily guarded.

AllanIf you are an API customer anywhere in the world, building an app on OpenAI Stack, this watermark is turned off by default. You have to manually opt into it. Right. And for regular ChatGPT users, it's strictly limited to users in the European Union.

Speaker 1Aaron Powell It's purely a regional rollout to satisfy a specific regulatory headache, basically.

AllanAaron Powell Yeah, but the real kicker is the detector tool itself. Like the actual software that reads the loaded dice pattern to confirm if the text is AI generated.

Speaker 1Aaron Powell Oh, they are completely withholding it.

AllanAaron Powell Completely hidden. You can't use it, I can't use it.

Speaker 1Nobody can use it.

AllanIt's only being given to approved researchers and expert organizations on a strict case-by-case basis.

Speaker 1Aaron Powell Which makes zero sense if the goal is actual transparency.

AllanAaron Powell Exactly. And looking through their own technical report, this state-of-the-art hyper-exclusive system has a hilarious fatal flaw.

Speaker 1Oh, this is my favorite part.

AllanIf you take a 400-token passage of watermark text and you just manually replace 10% of the words with synonyms.

Speaker 1Just 10%.

AllanThe detection accuracy drops from 92% down to 66%. That's terrible. And if you swap out 25% of the words, it plummets to a completely useless 17%.

Speaker 1Aaron Powell It just washes right out.

AllanYou barely even have to rewrite the text. So wait, it gets better. They built this state-of-the-art mathematically perfect bank fault, right?

Speaker 1Yeah.

AllanBut if the bank robber just wears a slightly different hat, like using a synonym, the security alarm completely ignores them.

Speaker 1Aaron Powell It really is the perfect provenance solution for a company that wants to claim regulatory compliance without um the inconvenience of anyone actually being able to verify the output.

AllanAaron Powell Right. If the public can't access the detector, who's going to know it failed?

Speaker 1Aaron Powell Exactly. And the contrast here is just staggering because while OpenAI's text engineers are doing all these delicate statistical token couplings to protect their own output, their image generator is out in the wild executing literal signature forgery across the internet.

AllanOh, right. The cartoons. So an investigation by Andrew Deck at Neiman Lab really blew the lid off this, didn't it?

Speaker 1It did. Deck documented over 15 New Yorker cartoonists whose authentic signatures and pen names were stamped onto AI-generated cartoons.

AllanNo permission asked.

Speaker 1No permission, no royalties paid, nothing.

AllanJust hijacked identities. And the human toll of this is devastating when you hear from the artists themselves. Like veteran cartoonist Joe Dater had this quote that literally stopped me in my tracks.

Speaker 1What did he say?

AllanHe said, I've had people hack my credit card. That feels like less of a violation than this. When they hacked my credit card, they didn't dress up like me.

Speaker 1Oh wow. That is such a visceral reaction.

AllanRight.

Speaker 1And the Neiman Lab investigation highlighted several cases exactly like that. There's Emily Flake, whose pen name E Flake was attached to these generic, soulless, single panel cartoons. She likened seeing her signature on that AI slop to somebody attributing a fake quote to her. Like it's a fundamental misrepresentation of her voice.

AllanThe detail that really gets me, though, is Pat Burns.

Speaker 1Oh, the period.

AllanYes. Yeah. So Pat has been a cartoonist for decades. And since 1998, he has placed a very specific period at the end of his signature. It reads P.Burns. Right. He describes that period as a personal mark, like a little gift to the reader. And ChatGPT meticulously forged that exact 1998 period across dozens of fakes.

Speaker 1Aaron Powell And we really need to look at how an AI actually does that, because it's not just, you know, copying and pasting a JPEG of his signature from Google Images.

AllanRight. It doesn't work like a collage.

Speaker 1Trevor Burrus, Jr. Exactly. A diffusion model starts with pure static, like old TV snow, and it's trained to slowly remove that noise to reveal an image based on its training data. Aaron Powell Okay.

AllanSo when it ingested thousands of New Yorker cartoons, it didn't learn this is Pat Byrne's name. No. It just learned that in the bottom right corner of these types of images, the pixels usually resolve into this specific squiggly shape, complete with a dotted at the end.

Speaker 1Aaron Powell Precisely. It learned the exact geometry of a human habit as a structural requirement of the image itself. But AI companies often defend this. They say, look, we're just teaching a machine an art style. We've seen AI mimic Van Gogh or Picasso for years. It's just an aesthetic.

AllanAaron Ross Powell Okay, but here's the thing. A style is an aesthetic. A signature is a certificate of authenticity. Trevor Burrus, Jr.

Speaker 1Exactly.

AllanTrevor Burrus It's the artist saying directly to the audience, I endorse this, I made this. Aaron Powell Yeah.

Speaker 1Jason Adam Katzenstein, another cartoonist, called the faked signatures slop that's pretending to be me.

AllanAaron Powell That's a perfect description.

Speaker 1Aaron Powell And he pointed out the much darker economic reality here, too. He believes the ultimate goal of these systems is to deal a death blow to labor.

AllanAaron Powell Which isn't an exaggeration because you can see that labor displacement happening right now.

Speaker 1Oh, everywhere.

AllanYou've got legacy publications like The Baltimore Sun firing veteran Pulitzer Finalist staff members like Kevin Callower, just so they can run cheap AI-generated opinion illustrations instead.

Speaker 1Aaron Powell It's incredibly bleak.

AllanAaron Powell And the hypocrisy of where these fakes are ending up is staggering. Like some of these forged cartoons even feature the trademarked New Yorker logo.

Speaker 1Right.

AllanBut Condy Nass, the parent company, signed a massive licensing deal with OpenAI in 2024. That deal explicitly prohibited training AI models on their cartoons or utilizing publisher logos.

Speaker 1That's a blatant violation of a signed contract.

AllanSo what did OpenAI do when they got caught?

Speaker 1When OpenAI was notified about this by Neiman Lab, their response was um incredibly passive.

AllanShocker.

Speaker 1They basically just added a generic prompt warning to Chat GPT saying requests, quote, may violate our guardrails concerning similarity to third-party content.

AllanOh, how helpful.

Speaker 1Right. But the system just kept outputting signed cartoons anyway.

AllanAaron Powell Of course it did. Because given how fast they scaled these models, they had to know they were scraping copyrighted material. I mean, there's no way legal didn't leave a paper trail.

Speaker 1Oh, there's a paper trail.

AllanLike how exactly doesn't AI get a hold of Pat Byrne's 1998 piece in the first place?

Speaker 1By eating everything on the internet and knowing exactly what they were doing while they did it.

AllanAaron Powell Do we have proof of that?

Speaker 1We do. In September 2026, court records were unsealed from the Authors Guild v. OpenAI lawsuit. And these internal documents from the early days of model training are just incredibly revealing.

AllanOkay. Lay the timeline out for me. What did they find?

Speaker 1Aaron Powell So in August 2019, internal Slack communications shows someone candidly admitting, and I quote, we trained GPT-3 on pirated stuff. No sharing that.

AllanNo way. No sharing that.

Speaker 1Yeah.

AllanThe golden rule of corporate liability. Just type it in the company Slack. That'll definitely keep it secure.

Speaker 1Aaron Ross Powell It gets worse, actually. A prominent researcher there, Sam McCandlish, admitted in these exact same messages that he was just worried about the optics.

AllanAaron Powell The optics.

Speaker 1Yeah. He specifically noted that headline coverage of their pirated training data landing on Hacker News would be unfortunate.

AllanAaron Powell Unfortunate. Which is the ultimate irony. Because when those exact unsealed documents came out in September 2026, they literally hit number one on Hacker News.

Speaker 1Aaron Powell Of course they did. The internet always wins. Always. But the internal culture was very clear about the path forward. In May 2020, then policy director Jack Clark wrote what has to be one of the most chillingly accurate predictions of this whole era.

AllanWhat did he say?

Speaker 1He wrote There will be a point where a bunch of artists express worry about what we're doing here, and we'll likely ignore their concerns and release anyway.

AllanSo they knew exactly who was going to get hurt, and they just what budgeted for it?

Speaker 1Basically. And by June 2022, as public scrutiny and litigation started really heating up, OpenAI launched an internal effort code named ProjectClear.

AllanProject Clear. Sounds ominous.

Speaker 1The entire goal was to secretly delete its pirated libgen datasets, which are these massive repositories of stolen books and media, from company servers. Exactly. That's the technical nightmare of Project Clear. The model doesn't store the actual books, right? It stores the statistical relationships between words that it learned from those books.

AllanRight.

Speaker 1To delete LibGen, they were essentially trying to unbake a cake. They had to try and isolate and excise the influence of those specific pirated texts from billions of neural weights, all while keeping it completely quiet.

AllanDeleting the evidence while trying not to break the product. That is wild.

Speaker 1Aaron Ross Powell And this culture wasn't just isolated to open AI either.

AllanReally?

Speaker 1Not at all. Anthropic, which was actually founded by ex-openAI executives like Jack Clark, faced the exact same reckoning. In 2025, Anthropic was forced into a record-breaking $1.5 billion settlement after it was caught downloading pirated book datasets to train its clawed models.

AllanA billion and a half dollars.

Speaker 1Yeah. We are looking at an entire industry culture that viewed human creation as raw material, erased the evidence under secret codenames, and dismissed the destruction of artists' livelihoods as acceptable economic disruption.

AllanAcceptable economic disruption.

Speaker 1Which is a direct quote from another open AI employee, by the way.

AllanJeez. Think about what this means for you, the person listening right now. If someone clones your voice or your face or your writing style for a meme, what recourts do you actually have?

Speaker 1Almost none.

AllanBut if the theft is this painfully obvious, and we have the slack messages and the forged 1998 periods, why can't the cartoonists just band together and sue them into oblivion?

Speaker 1You'd think it would be a slam dunk case, right? But the legal reality is a massive catch-22.

AllanOkay, explain.

Speaker 1James Grimelman, a law professor at Cornell, breaks this down really well. Under standard copyright law, you are protecting specific individual images. You cannot copyright a general artistic style.

AllanOkay, that makes sense.

Speaker 1So when the AI synthesizes a general New Yorker vibe, it skirts direct copyright infringement. And Grimmelman notes that attribution, the actual act of putting someone's name on a work, actually plays a very tiny role in the legal test for fair use.

AllanAaron Powell Okay, sure. The vibe isn't copyrighted. I get that. But what about the signature itself? That is a person's actual identity. Can't they use like identity theft or publicity laws?

Speaker 1Aaron Ross Powell They can try state right of publicity laws. Those are explicitly designed to protect your name and personal identity from unauthorized use. Right. But here is the massive loophole. To win a right of publicity case, you usually have to prove that the fake identity was used for a commercial purpose. Uh-huh. Like selling a product or running an ad campaign.

AllanAaron Powell Oh, I see exactly where this is going. The person who prompted the Dolly Parton cartoon was just a fan.

Speaker 1Exactly.

AllanTrevor Burrus They generated it for free and posted it to X for likes.

Speaker 1Yep. Because it's non-commercial fan activity, it is incredibly hard to establish liability under current publicity statutes.

AllanAaron Powell That is so frustrating.

Speaker 1Aaron Ross Powell The technology has completely outpaced the legal framework. Think about it. The AI company built the forgery tool, but they didn't push the button to make that specific cartoon.

AllanRight.

Speaker 1The user pushed the button, but they aren't making money off it. So who is legally responsible?

AllanNo one. This is simultaneously impressive and completely ridiculous. Truly. So the AI can steal your exact identity, perfectly forge your handwritten signature, and literally destroy your professional credibility.

Speaker 1Uh-huh.

AllanBut because a user posted it as a free meme, the multi-billion dollar company that built the forgery machine in the first place is legally untouchable.

Speaker 1Aaron Powell That's the reality. And if we pull all these threads together, the thematic synthesis is pretty stark. Yeah. On one hand, we have granted artificial intelligence the most complex, optimal transport mathematics imaginable, just so it can verify its own synthetic provenance down to the specific token. We have bent over backwards to build a microscopic fingerprint for the machine.

AllanWhile the oldest provenance technology in human history, a handwritten signature, like a person's literal mark of existence, is treated as disposable open source training data.

Speaker 1Yep.

AllanThe machine gets mathematical certainty, the human gets acceptable economic disruption.

Speaker 1Exactly.

AllanExactly.

Speaker 1Stepping back and looking at this whole deep dive, it really makes you think about what this says about us as a society. It really does. We are pouring billions of dollars into building tools that are supposed to, you know, restore trust in the digital age, but these tools are actually just mirrors.

AllanMirrors.

Speaker 1They reflect our priorities. And right now they show that we are obsessed with authenticating the machines while offering zero protection to the humans that the machines are literally built out of.

AllanAaron Powell It's a profound imbalance. And honestly, if you ever find yourself needing to rob a bank, remember the text green loophole? Just wear a synonym.

Speaker 1Just wear a synonym. The alarm will never go off.

AllanNever.

Speaker 1But I want to leave you with one final slightly provocative thought to mull over after we wrap up here today. We are watching AI perfectly replicate the visual intent of a signature, the curves of the letters, the specific period from 1998. But the AI entirely lacks the consciousness of what a signature actually means.

AllanBecause to a human, a signature is a binding promise. It's an oath.

Speaker 1Right. It means something. If our massive tech companies are perfectly fine normalizing this kind of hollow algorithmic forgery and comic strips and pearly gates jokes today, what happens tomorrow? What happens when we start relying on these exact same AI systems to authenticate legally binding contracts or medical records or international treaties? If the fingerprint lies, what happens to the hand?

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Frontier AI Turns PHP Memory Bugs Into A Push Button Exploit

    Oct 22, 202622 minAudio

    A decade-old flaw in PHP flips from “too hard to exploit” into a real-world remote code execution chain once frontier AI enters the picture. We break down how the memory bugs work, why the blast radius hits everything...

    Credit the people on this episode

  2. When Google’s Cloud Fell: How a Power Dip Broke Banks, Cards, and Delivery

    Oct 20, 202622 minAudio

    Google Cloud outage, cloud resilience, data center power failure. Yet a flicker inside a London Google Cloud availability zone helps trigger a cascading cloud outage that leaves people staring at declined cards, frozen...

    Credit the people on this episode

  3. Anthropic's Doomsday IPO: We Warned You. Now Buy the Stock

    Sep 30, 202622 minAudio

    A normal IPO filing is supposed to answer one question: why is this company a safe place to put serious money? So what do we do with an AI prospectus that spends pages warning about existential risk, models that might...

    Credit the people on this episode

  4. Anthropic's "Ethical" AI Just Got Caught Snooping

    Jul 9, 202623 minAudio

    A tiny punctuation mark shouldn’t be able to start a scandal, but that’s exactly what makes this story so unsettling. We dig into reports that Anthropic’s developer tool, Claude Code, quietly swapped a standard...

    Credit the people on this episode

  5. The Emperor's New Code: Inside the Boardroom Delusion of the AI Hype Cycle

    Jul 6, 202621 minAudio

    A half-billion-dollar AI bill from a missed checkbox sounds like a one-off punchline until you follow the thread. We start with that Axios-style nightmare scenario and pull it into something far more unsettling: when a...

    Credit the people on this episode