Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

The Human in the Loop

The Human in the Loop artwork

AI Ships Faster Than Anyone Can Review It

May 31, 202622 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Meta says AI writes 80% of new code. Their own reviewers can't keep up with their own AI. Straight from their engineering blog. They built RADAR to auto-review low-risk diffs because "the share of diffs receiving timely review has declined." Their words. AI-generated code outpaced human review capacity. Read that with the rest of the week's news. Cognition says Devin merged 7x more PRs year-over-year. AI-written commits inside customer codebases jumped from 16% to 80%. Anthropic shipped Opus 4.8 on Wednesday, and every IDE, gateway, and agent runner had it the same day. They also disclosed a $47B revenue run-rate. The "is this a real business" debate is over. But here is what keeps coming back to me: Shipping more code faster is only a win if the systems that catch problems scale at the same rate. This week, the evidence says they aren't. A new arXiv study of 20,574 real coding-agent sessions documents how often agents do something other than what was asked. ITBench-AA, the first serious benchmark for agentic IT work, scored every frontier model below 50%. Adoption is real. The guardrails are not. This week's episode of The Human in the Loop covers all of it: the shipping wave, the cost-control backlash starting inside eng departments, and why ITBench-AA matters more than the score suggests.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Who Understood the Code? Rethinking Engineering Interviews

    Nov 1, 202619 minAudio

    If an agent can solve your interview puzzle in seconds, what are you actually measuring? This week an Anthropic engineering manager, Felix Rieseberg, wrote that he stopped using coding interviews for his team. Claude...

    Credit the people on this episode

  2. Git’s best feature was slowing down Anthropic’s machines

    Oct 18, 202624 minAudio

    Git was built for a developer on a train. Every copy works offline. That was the point. Anthropic says its merges went from about 2,500 a day in May to over 30,000 by September. Most of the clients are now agents and CI...

    Credit the people on this episode

  3. The 49% Problem: Who Reviews What the Agents Write?

    Oct 11, 202623 minAudio

    AI coding agents wrote more code. Human review time went up 49%. That comes from a study covered by Ars Technica. I could not trace the original, so treat the numbers as reported. Lines of code: up about 30%. Pull...

    Credit the people on this episode

  4. Can You Explain It?" Airbnb's One Rule for AI-Written Code

    Oct 4, 202622 minAudio

    Airbnb's CTO says 60% of its code is now AI-written. He has one rule for it. Ahmad Al-Dahle shared this in a Latent Space interview this week. Features shipped: up nearly 80% year over year. Pull-request throughput: up...

    Credit the people on this episode

  5. Your AI Agent Doesn't Know What Its Code Does

    Sep 27, 202618 minAudio

    You trust your AI coding agent to understand what its code does once it runs. A new benchmark says that trust is misplaced most of the time. Researchers built SWE-Flux to test it: 480 questions across 12 real Python...

    Credit the people on this episode