Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Linear Digressions

Linear Digressions artwork

The Lottery Ticket Hypothesis

Feb 23, 202020 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Recent research into neural networks reveals that sometimes, not all parts of the neural net are equally responsible for the performance of the network overall. Instead, it seems like (in some neural nets, at least) there are smaller subnetworks present where most of the predictive power resides. The fascinating thing is that, for some of these subnetworks (so-called “winning lottery tickets”), it’s not the training process that makes them good at their classification or regression tasks: they just happened to be initialized in a way that was very effective. This changes the way we think about what training might be doing, in a pretty fundamental way. Sometimes, instead of crafting a good fit from wholecloth, training might be finding the parts of the network that always had predictive power to begin with, and isolating and strengthening them. This research is pretty recent, having only come to prominence in the last year, but nonetheless challenges our notions about what it means to train a machine learning model.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. How to Stay Relevant as a Data Scientist in the Age of AI | Interview with Dr. Jon Krohn

    Nov 16, 202644 minAudio

    Staying relevant as a data scientist when AI is reshaping the field isn't just a career question - it's an existential one. Dr. Jon Krohn, neuroscientist-turned-data-science-educator, podcaster, and consultant, sits...

    Credit the people on this episode

  2. GPU Architecture for LLM Inference: Memory, Bandwidth and Hardware Explained

    Nov 9, 202620 minAudio

    GPUs are the engines powering every AI conversation you've ever had - but what's actually happening inside those chips when an LLM generates a response? Understanding GPU fundamentals turns out to be surprisingly useful...

    Credit the people on this episode

  3. KV Cache Fundamentals

    Nov 2, 202624 minAudio

    The unsung hero of every LLM conversation you've ever had? A clever optimization called the KV cache. Self-attention is computationally brutal - every token attending to every other token means costs that scale as N²,...

    Credit the people on this episode

  4. AI Inference 101

    Oct 26, 202628 minAudio

    Everyone talks about training the model - the data, the compute, the alignment magic. But what happens after the weights are frozen and millions of users start hammering it with prompts? That's inference, and it's a...

    Credit the people on this episode

  5. Back Next Week

    Oct 19, 202618 secAudio

    Back Next Week by Katie Malone

    Credit the people on this episode