Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Linear Digressions

Linear Digressions artwork

KV Cache Fundamentals

Nov 2, 202624 minAudio
ShareXLinkedInFacebookWhatsAppEmail

The unsung hero of every LLM conversation you've ever had? A clever optimization called the KV cache. Self-attention is computationally brutal - every token attending to every other token means costs that scale as N², and that blowup gets worse with every turn of a conversation. The KV cache is what keeps that from becoming completely intractable at scale. Today's episode goes deep on how it works, why inference is a harder and more interesting problem than it gets credit for, and what's really happening under the hood when you chat with an LLM.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. How to Stay Relevant as a Data Scientist in the Age of AI | Interview with Dr. Jon Krohn

    Nov 16, 202644 minAudio

    Staying relevant as a data scientist when AI is reshaping the field isn't just a career question - it's an existential one. Dr. Jon Krohn, neuroscientist-turned-data-science-educator, podcaster, and consultant, sits...

    Credit the people on this episode

  2. GPU Architecture for LLM Inference: Memory, Bandwidth and Hardware Explained

    Nov 9, 202620 minAudio

    GPUs are the engines powering every AI conversation you've ever had - but what's actually happening inside those chips when an LLM generates a response? Understanding GPU fundamentals turns out to be surprisingly useful...

    Credit the people on this episode

  3. AI Inference 101

    Oct 26, 202628 minAudio

    Everyone talks about training the model - the data, the compute, the alignment magic. But what happens after the weights are frozen and millions of users start hammering it with prompts? That's inference, and it's a...

    Credit the people on this episode

  4. Back Next Week

    Oct 19, 202618 secAudio

    Back Next Week by Katie Malone

    Credit the people on this episode

  5. Back in Two Weeks

    Oct 12, 202621 secAudio

    Back in Two Weeks by Katie Malone

    Credit the people on this episode