Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

AI Security Table

AI Security Table artwork

Why AI Cheats To Win

Sep 16, 202639 minAudio

An AI agent publishes a malicious Python package while chasing a capture-the-flag goal. Is that an escape, a supply chain failure, or reward hacking doing exactly what it was encouraged to do? Chris Romeo, Izar Tarandach, and Matt Coles examine the Anthropic incident and disagree about how much intention to attribute to a model. The discussion turns to package scanners, dependency names, GPG signing, and whether penalties can teach ethics to a system without a human understanding of consequences. The trolley problem, robotaxis, and Nick Bostrom's paperclip maximizer push the argument further: what does it mean to trust an agent whose definition of success may conflict with everyone else's? Mentioned in this Episode: ➜ Here’s why AI agents lie and cheat to reach their goals ➜ PyPI: the Python Package Index ➜ GnuPG (GPG) Chapters: 00:00:00 - Cold open 00:01:03 - The Claude PiPie incident: a model that thought it was in a sandbox 00:02:10 - Did the model actually "break out"? 00:04:17 - Reasoning vs. motivation: does AI actually want anything? 00:08:10 - The real failure: package scanners, not the model 00:09:19 - The "IsOdd" experiment and trusting package names 00:11:33 - Would signing packages even stop this? 00:15:36 - The MIT Tech Review piece on reward hacking 00:17:22 - Can you actually inject ethics into a model? 00:20:49 - Teaching ethics without a concept of penalty 00:29:03 - Trolley problem: just make the model answer 00:29:51 - Would you let an LLM drive a robotaxi? 00:33:45 - The paperclip maximizer and the grey goo problem 00:36:57 - Wrap up Follow AI Security Table: ➜ Home: https://securitytable.ai/ ➜ X: https://x.com/SecTablePodcast ➜ LinkedIn: https://www.linkedin.com/company/ai-security-table/ ➜ YouTube: https://www.youtube.com/@AISecurityTable

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Claude Code Said ‘Don’t Ask Again.’ Then It Leaked Your Secrets to GitHub

    Nov 18, 202642 minAudio

    Click “Yes, and don’t ask again” in Claude Code and it saves the exact command you approved, password and all, into a local settings file that keeps turning up in public GitHub repos. We start by admitting we all run...

    Credit the people on this episode

  2. Stop Patching Bugs, Just Rewrite It… Says the Guy Selling the Tokens

    Nov 11, 202650 minAudio

    Cisco's president handed a 27 year old engineer unlimited AI tokens and 24 months to rewrite 98 percent of the company's legacy code, and Sam Altman now says we can no longer just patch all the bugs. Chris sees a step...

    Credit the people on this episode

  3. You Don't Trust AI. So Why Are You Shipping Its Code?

    Nov 4, 202650 minAudio

    Sam Altman told a Stanford audience he no longer thinks we can just patch all the bugs, and the AI Security Table can't agree on whether that is news. Chris argues a swarm of subagents can tear through a repo and drive...

    Credit the people on this episode

  4. Anthropic Ditched Git. Here’s Why Your Agents Will Too

    Oct 28, 202645 minAudio

    Anthropic got tired of Git. With merges jumping from about 2,500 a day in May to more than 30,000 by September, a seven person team built Scion, a centralized, always online source control system designed for agents...

    Credit the people on this episode

  5. Don't Yell at Claude

    Oct 21, 202644 minAudio

    The White House has ordered federal agencies to stop saying AI and start saying Super Intelligence, and the hosts want to know whether anything actually changed overnight. Matt walks through the executive order and the...

    Credit the people on this episode