Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

The Human in the Loop

The Human in the Loop artwork

Ponytail Activation 0%

Aug 2, 202621 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Advertised 54%. Measured 15%. Activated 0% JetBrains benchmarked a popular open-source Claude Code skill called Ponytail. It pushes the agent to write less code. The authors has promised 54% less code, 20% less cost. 80 paired tasks. Sonnet 5. Every trial audited. Reality: 15.4% less code, 10.3% less cost. No quality difference either way. Smaller than the label, but real. Then the part that matters more. Installed passively, the skill showed zero self-activation. It did nothing at all unless the ruleset was force-injected into the session. A tool with measurable benefits, sitting there inert. I have not run this one myself, so this is JetBrains' data and not mine. But the implication travels past one skill. If you rolled out agent skills across your team and measured adoption by install count, you measured nothing. Install count says the file exists. It says nothing about whether it ran. Full breakdown in this week's episode of The Human in the Loop. Link in the comments. #ClaudeCode #AgentSkills #AIAdoption

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Who Understood the Code? Rethinking Engineering Interviews

    Nov 1, 202619 minAudio

    If an agent can solve your interview puzzle in seconds, what are you actually measuring? This week an Anthropic engineering manager, Felix Rieseberg, wrote that he stopped using coding interviews for his team. Claude...

    Credit the people on this episode

  2. Git’s best feature was slowing down Anthropic’s machines

    Oct 18, 202624 minAudio

    Git was built for a developer on a train. Every copy works offline. That was the point. Anthropic says its merges went from about 2,500 a day in May to over 30,000 by September. Most of the clients are now agents and CI...

    Credit the people on this episode

  3. The 49% Problem: Who Reviews What the Agents Write?

    Oct 11, 202623 minAudio

    AI coding agents wrote more code. Human review time went up 49%. That comes from a study covered by Ars Technica. I could not trace the original, so treat the numbers as reported. Lines of code: up about 30%. Pull...

    Credit the people on this episode

  4. Can You Explain It?" Airbnb's One Rule for AI-Written Code

    Oct 4, 202622 minAudio

    Airbnb's CTO says 60% of its code is now AI-written. He has one rule for it. Ahmad Al-Dahle shared this in a Latent Space interview this week. Features shipped: up nearly 80% year over year. Pull-request throughput: up...

    Credit the people on this episode

  5. Your AI Agent Doesn't Know What Its Code Does

    Sep 27, 202618 minAudio

    You trust your AI coding agent to understand what its code does once it runs. A new benchmark says that trust is misplaced most of the time. Researchers built SWE-Flux to test it: 480 questions across 12 real Python...

    Credit the people on this episode