Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Practical AI

Practical AI artwork

Large models on CPUs

May 2, 202338 minAudio

Model sizes are crazy these days with billions and billions of parameters. As Mark Kurtz explains in this episode, this makes inference slow and expensive despite the fact that up to 90%+ of the parameters don’t influence the outputs at all. Mark helps us understand all of the practicalities and progress that is being made in model optimization and CPU inference, including the increasing opportunities to run LLMs and other Generative AI models on commodity hardware. Sponsors: Fastly - Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com Fly.io - The home of Changelog.com - Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs. Featuring: Mark Kurtz - LinkedIn, X Daniel Whitenack - Website, GitHub, X Show Notes: Neural Magic SparseML SparseZoo Neural Magic Scales up MLPerf™ Inference v3.0 Performance With Demonstrated Power Efficiency; No GPUs Needed Deploy Optimized Hugging Face Models With DeepSparse and SparseZoo SparseGPT: Remove 100 Billion Parameters for Free Upcoming Events: Register for upcoming webinars here!

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Hosts

  • Daniel Whitenack Host Feed-stated

Guests

  • Mark Kurtz Guest Feed-stated

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Building Human-Centered AI

    Nov 19, 202645 minAudio

    How can enterprises adopt AI and agentic systems while keeping people, business value, and trust at the center? Chris is joined by Dr. Gautham Pallapa, Principal Director of AI and Innovation at Scotiabank and author of...

    Credit the people on this episode

  2. Implementing Federated Learning to enhance Agentic AI

    Nov 13, 202644 minAudio

    How can AI models learn from private data without compromising privacy? Chris is joined by Lorenzo Sani, Research Scientist at Flower Labs and PhD researcher at Cambridge, to explore federated learning and...

    Credit the people on this episode

  3. Abundant Intelligence at the Midwest AI Summit Part 2

    Nov 5, 202648 minAudio

    How do organizations decide what AI should do, where to set boundaries, and when to keep humans in control? In this special episode, we bring you three more talks from the Midwest AI Summit, exploring how organizations...

    Credit the people on this episode

  4. Abundant Intelligence at the Midwest AI Summit

    Oct 29, 202654 minAudio

    In this special episode, we bring you highlights from three talks at the Midwest AI Summit. Heidi Porter explores how to build reliable AI-augmented workflows using the right tools, evaluations, and guardrails. Kevin...

    Credit the people on this episode

  5. Exploit Development, Agentic Vulnerability Discovery, and Fake PoCs

    Oct 22, 202645 minAudio

    AI is changing cybersecurity, accelerating vulnerability research and exploit development while raising new questions about reliability, risk, and guardrails. Daniel and Chris talk with Spencer McIntyre, Director of...

    Credit the people on this episode