Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Super Data Science: ML & AI Podcast with Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn artwork

1028: The Chip Built for Agentic AI Inference, with SambaNova's Anton McGonnell

Sep 18, 202630 minAudio

In Episode #1028, Anton McGonnell (VP of Product at SambaNova) joins Jon Krohn to explain why the chips running most AI inference today were never designed for the job. Agentic AI has changed the computational profile of inference, with much larger inputs and far heavier caches feeding the token generation that follows, and that shift has exposed where GPU architecture struggles. SambaNova has raised over $2 billion to build an alternative, the reconfigurable dataflow unit, which lays a whole model out spatially across the chip rather than executing it kernel by kernel. In this episode, Anton discusses why the speed that matters is payback, and how speed and concurrency are what turn a fixed hardware cost into a six-month payback. He also walks through the trade-off every inference provider faces between speed per user and throughput per chip, what the RDU architecture changes about scaling and data center deployment, the economics of the new SN50, and why four out of five AI infrastructure leaders say they would pay a premium for faster tokens. Additional materials: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.superdatascience.com/1028⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information. In this episode you will learn: (00:02:41) Why agentic AI is reshaping inference workloads (00:08:39) How SambaNova's RDU differs from a GPU (00:17:54) The economics of the SN50

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. 1046: Weighing a Feather: How to Design A/B Tests That Work, with Billur Engin

    Nov 20, 202625 minAudio

    In Episode #1046, Billur Engin (who leads data science and analytics at Prolific) joins Jon Krohn to explain how to design experiments that actually teach you something. Too many A/B tests, she says, are like bathroom...

    Credit the people on this episode

  2. 1045: How AI Chips Are Built, with Benjamin Hein

    Nov 17, 202656 minAudio

    In Episode #1045, Ben Hein (CEO of the Electronics business at Merck KGaA, Darmstadt, Germany) joins Jon Krohn to explain how the chips behind the AI boom are actually made. His company is 358 years old, began as a...

    Credit the people on this episode

  3. 1044: In Case You Missed It in October 2026

    Nov 13, 202632 minAudio

    In this month's episode of ICYMI, Jon Krohn moves from the fears that surround AI to the work it makes possible when we use it well. Hear from Richard Socher, Carly Taylor, Jeff Hancock, Kate Niederhoffer and John...

    Credit the people on this episode

  4. 1043: AI-Powered Precision Nutrition, with Viome’s Dr. Guru Banavar

    Nov 10, 20261 hr 13 minAudio

    In Episode #1043, Dr. Guru Banavar (Founding CTO of Viome) joins Jon Krohn for a technical masterclass on the biology inside your gut. Viome has assembled what Guru calls the largest dataset on human biology ever...

    Credit the people on this episode

  5. 1042: Building for the Agent-Native App Store, with Salesforce’s Tyler Carlson

    Nov 6, 202637 minAudio

    #Sponsored, SVP, Head of Product, AgentExchange and Partner Experience, Tyler Carlson, returns to talk to Jon Krohn about building for an agent-native app store. Since Tyler’s last appearance in Episode 956, Salesforce...

    Credit the people on this episode