Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

TestGuild Automation Podcast

TestGuild Automation Podcast artwork

AI Agents Passed the Tests. Real Devices Didn't. with Nick Metcalfe

Sep 29, 202641 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Cloud device farms make it easy to get started with real device testing. But what happens when you need to scale? Costs climb fast, you wait in a queue for the devices you need, and you lose visibility into what's actually happening when a test fails. In this episode, Nick Metcalfe, Head of Technical Sales Engineering at Cambrionix, joins Joe to explain why more testing teams are bringing their device labs in house. Nick shares how one team was quoted roughly a million dollars a month to outsource their real device testing, then cut that cost to around $300K by building their own lab. And cost is only part of the story. You'll discover why an in house device lab can outperform a cloud lab: Lower cost at scale. Cloud pricing works for small test runs, but at enterprise scale an in house lab can save hundreds of thousands of dollars every month. No waiting in line. Your devices are yours, available whenever you need them, with no queue and faster turnaround. Full control and visibility. See exactly what's happening at the hardware level, down to the millisecond a USB controller reboots, so you can tell a real bug from an infrastructure failure. Test the devices your users actually have. Keep legacy phones, budget devices, and hand me downs in your lab, just like Snap does to support its user base across 12,000+ devices and 700,000 tests a day. Test custom hardware. Point of sale systems, VR headsets, smart devices, and prototypes that a cloud vendor simply won't have. Automate the hands on work. Reboot devices, recover from restore mode, and put 16 Apple devices into DFU mode at once, all through an API that plugs into your CI pipeline. Protect your device investment. Hold batteries at a set charge level to reduce wear and extend device life for years. Nick also covers the honest tradeoffs, including the setup and resources an in house lab requires, and why a hybrid approach of emulators first and real devices second gives you the most confidence before release. Plus, his number one tip for building a lab: do it once, do it right. If your team is scaling mobile test automation, fighting flaky tests, or questioning your cloud device farm bill, this episode is for you.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Robot Framework Test Automation: Automate Web, Desktop, Mobile, and Embedded Systems

    Nov 17, 202636 minAudio

    What makes Robot Framework flexible enough to test medical devices, low code bots, monitoring systems, and embedded hardware, all with one framework? In this RoboCon 2027 special, Joe Colantonio talks with four leaders...

    Credit the people on this episode

  2. Playwright at Scale: Run 500 Tests in Minutes

    Nov 10, 202637 minAudio

    Is your Playwright suite slowing you down instead of speeding you up? In this episode, Oliver Stenbom, cofounder of Endform, shares what really happens when teams try to scale Playwright from a handful of tests to...

    Credit the people on this episode

  3. How to Prove Your Release Is Ready When AI Writes the Code with Ashwin Kondapalli

    Nov 3, 202639 minAudio

    How to Prove Your Release Is Ready When AI Writes the Code? Can you prove your software is ready to ship, or does that answer still depend on a meeting and a spreadsheet? Want to see if your team can prove a release is...

    Credit the people on this episode

  4. Why Your AI Tests Pass When Your Features Fail with Donovan Brady

    Oct 27, 202649 minAudio

    Why Your AI Tests Pass When Your Features Fail? Donovan Brady of Leapwork shows how to make AI tests traceable, cheaper to run, and safe to self-heal. 👉 Try Leapwork Play (free tier):...

    Credit the people on this episode

  5. Scary Test Automation POCs Made Easy: Halloween Special With Paul Grossman

    Oct 20, 202642 minAudio

    Running a proof of concept on test automation tools can feel like a horror movie. Vendors promise everything, your boss wants answers fast, and you don't know what you don't know. In this Halloween special, Paul...

    Credit the people on this episode