Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Learning Bayesian Statistics

Learning Bayesian Statistics artwork

Can Neural Networks Replace MCMC?

Oct 26, 20264 minAudio

Today’s clip is from Episode 167 of Learning Bayesian Statistics, featuring Paul-Christian Bürkner and Steve Bronder, recorded live at StanCon 2026 in Uppsala, Sweden. Paul Bürkner, creator of brms and a leading researcher behind BayesFlow, explains how neural networks can replace MCMC for Bayesian inference. Instead of evaluating likelihood and prior densities, the networks are trained on simulated data, so you can do simulation-based inference (SBI) even when the likelihood is intractable. Once trained, they give you amortized, real-time posteriors for new datasets, and they have made SBI scalable well beyond what approximate Bayesian computation could handle. Paul is also candid about the limits. Neural methods come with unstable training and no guarantees, and they can be unreliable on out-of-distribution or misspecified data, which is why MCMC is still the more trustworthy option wherever you can apply it. He ends with how his group is building diagnostics to flag when a neural posterior can’t be trusted. Full discussion here Support & Resources → Support the show on Patreon → Bayesian Modeling Course (first 2 lessons free): Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work!

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show.

speaker-0: Have you now that you've been working on that for a few years now, have you seen any any interesting applications of these kind of methods, especially using base flow, of course?

speaker-1: Yeah, we do have a lot of special. the likelihood density is available or easily available and we just, but we can efficiently

speaker-2: From these models. So we then in simulation base Bayesian inference approximate

speaker-1: which we did have like.

speaker-2: Bayesian computation for decades, but it didn't really scale beyond.

speaker-1: beyond a few parameters. And neural networks really humiliated when they came in. Made

speaker-2: And

speaker-1: SBI really feasible and scalable. And as a side project, we got this amortization property that gives us real-time Bayesian inference.

speaker-2: Mm and so all the kind of models models are sometimes

speaker-1: like physical models from psychology or biology or chemistry where we don't have these analytic likelihood densities, which you don't see at Stencon, of course. For these, the simulation-based methods are really like the state of the art, if you will. But of course, they would still not reach the reliability specifically for out of distribution data for like data for the modified.

speaker-2: Yeah,

speaker-1: then MCMC would have in comparison if we could apply MCMC. So a lot of the core applications where this has been really successful is in places where we cannot efficiently do MCMC in the first place. Because if we did, probably MCMC would still win at least in the accuracy and the trustworthiness that we get out of it. It's kind of my goal to improve that trustworthiness of the neural methods. that would be

speaker-2: We get faster inference, but

speaker-1: but similarly accurate.

speaker-0: Yeah. Can you

speaker-2: Is there is there methods yet or measures you can use to say, well this didn't quite work as we wanted it to? So

speaker-1: seeds

speaker-2: in in a way, yes. But it only works and you know that giving

speaker-1: I mean, they're just.

speaker-2: for context, it only works of course if we are simulating from the exact same model that then we are essentially fitting, right? but of course

speaker-1: If that works.

speaker-2: Let's say with MCMC implemented in STEN or wherever, then we have reasonable data, which may look a little different than the simulated data, you get an accurate

speaker-1: trust that also for real world data, still we

speaker-2: posterior approximation. So now in for for for kind of neural simulation based on amortized inference, of course we can run run L C. It's

speaker-1: be deeper.

speaker-2: more of a convergence diagnosis because we have been training the neural with actually this kind

speaker-1: exactly the

speaker-2: of simulated data. And we also know that on the neural networks are only good. to predict something that is kind of close enough to the data that is seen during training. We don't

speaker-1: So of SBC we may easily.

speaker-2: know if it will work for real world data. data

speaker-1: If the real world.

speaker-2: is somehow like out of distribution of the simulated data. Because our model

speaker-1: are usually misspecified.

speaker-2: in some way or form. So so that there's and this is actually what we're working with to trying to diagnose not when when the data when the model is misspecified for a given data, but when it is misspecified in way.

speaker-1: is that will affect our accuracy of the analytic posterior.

speaker-2: Which of course we don't know. So it's a it's a black box and the way we're doing it we are not we are we are we are having two neural networks, one is actually the posterior estimation and the other one does summarizing the in

speaker-1: data into ideally.

speaker-2: like informative ideally sufficient statistics. Check out

speaker-1: so we can check whether we are.

speaker-2: of distribution in the space of sufficient statistics. And if we are chances our posterior approximation will also be bad. Which gives us diagnostics and we have a paper soon coming out of of this that extends previous methods of ours and sometimes it works and sometimes it doesn't.

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. #169 Bayesian Workflow for Beginners, Alex at Princeton

    Nov 18, 202655 minAudio

    Support & Resources → Support the show on Patreon → Bayesian Modeling Course (first 2 lessons free) Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work!...

    Credit the people on this episode

  2. Why Start with Bayesian Multilevel Models for Repeated Measures?

    Nov 17, 20265 minAudio

    Today's clip is from Episode 168 of Learning Bayesian Statistics, featuring Santiago Barreda, who joins host Alex Andorra to discuss Bayesian multilevel models and why they are particularly useful for researchers...

    Credit the people on this episode

  3. Why Speech Researchers Should Give Bayesian Statistics a Shot

    Nov 12, 20264 minAudio

    Today's clip is from Episode 168 of Learning Bayesian Statistics, featuring Santiago Barreda, who joins host Alex Andorra to discuss Bayesian multilevel models and their applications in speech perception and...

    Credit the people on this episode

  4. #168 Speech Perception & Multilevel Models, with Santiago Barreda

    Nov 4, 20261 hr 26 minAudio

    Support & Resources → Support the show on Patreon → Bayesian Modeling Course (first 2 lessons free) Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work!...

    Credit the people on this episode

  5. How I built Learn Bayes Stats to the Top 1.5%

    Oct 28, 202623 minAudio

    This one is a bonus episode, where I was the guest, on a podcast called One Thing That Worked! Get all the insights and takeaways here. In the meantime, here is how they described it: In this insightful episode of One...

    Credit the people on this episode