Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

TestGuild Automation Podcast

TestGuild Automation Podcast artwork

Testing AI: Engineering Confidence in Non-Deterministic Systems with Jason Arbon

Aug 12, 202653 minAudio
ShareXLinkedInFacebookWhatsAppEmail

In this episode 600 of the TestGuild Automation Podcast, Joe Colantonio talks with Jason Arbon, founder of Testers.ai, Jank.AI and IcebergQA and author of the new book Testing AI: Engineering Confidence in Non-Deterministic Systems. Take Our 2027 Survey Now: https://testgld.link/27data Jason makes a case most testers have not heard yet. Coding is being absorbed by AI. Specification work is thinning out. Product, development, and test roles are converging into one. And when the music stops, the only seat left belongs to the person who can look at what the machine produced and make an evidence backed call on whether it ships. He calls that confidence engineering, and he argues it is not a rebrand of QA. It is what QA was always supposed to be. Along the way, Joe and Jason get into the containment problem and why alignment, not lockdown, is now the real safety goal. They dig into why testing cost scales quadratically, meaning ten times more generated code creates roughly a hundred times more testing demand. Jason also pushes back hard on skeptics of agentic testing, pointing out that almost nobody has run the obvious experiment of testing a site themselves for a week and comparing their results against what AI finds. You will also hear Jason's most practical piece of advice in the whole conversation. If you are not running the same suite five times against the same build and looking at the actual results, not just flake, you are not testing seriously in an AI world. Plus a detour into grokking, the Chinese Room, and Geoffrey Hinton, because it would not be a Jason Arbon episode without one. Listen up!

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Robot Framework Test Automation: Automate Web, Desktop, Mobile, and Embedded Systems

    Nov 17, 202636 minAudio

    What makes Robot Framework flexible enough to test medical devices, low code bots, monitoring systems, and embedded hardware, all with one framework? In this RoboCon 2027 special, Joe Colantonio talks with four leaders...

    Credit the people on this episode

  2. Playwright at Scale: Run 500 Tests in Minutes

    Nov 10, 202637 minAudio

    Is your Playwright suite slowing you down instead of speeding you up? In this episode, Oliver Stenbom, cofounder of Endform, shares what really happens when teams try to scale Playwright from a handful of tests to...

    Credit the people on this episode

  3. How to Prove Your Release Is Ready When AI Writes the Code with Ashwin Kondapalli

    Nov 3, 202639 minAudio

    How to Prove Your Release Is Ready When AI Writes the Code? Can you prove your software is ready to ship, or does that answer still depend on a meeting and a spreadsheet? Want to see if your team can prove a release is...

    Credit the people on this episode

  4. Why Your AI Tests Pass When Your Features Fail with Donovan Brady

    Oct 27, 202649 minAudio

    Why Your AI Tests Pass When Your Features Fail? Donovan Brady of Leapwork shows how to make AI tests traceable, cheaper to run, and safe to self-heal. 👉 Try Leapwork Play (free tier):...

    Credit the people on this episode

  5. Scary Test Automation POCs Made Easy: Halloween Special With Paul Grossman

    Oct 20, 202642 minAudio

    Running a proof of concept on test automation tools can feel like a horror movie. Vendors promise everything, your boss wants answers fast, and you don't know what you don't know. In this Halloween special, Paul...

    Credit the people on this episode