Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Platform Engineering Podcast

Platform Engineering Podcast artwork

From React to Dagster: Pete Hunt on Data, Infra, and AI-Ready Platforms

Jul 30, 202550 minAudio
  • Apple Podcasts
ShareXLinkedInFacebookWhatsAppEmail

Is Postgres actually a better message queue than Kafka? This provocative question is just one of many insights Pete Hunt shares in this conversation about data orchestration, platform engineering, and the evolution of infrastructure. Pete Hunt, CEO of Dagster Labs and former React co-founder at Facebook, brings his unique perspective from working at tech giants like Instagram and Twitter to discuss how different platform team approaches impact product development. Having witnessed both Facebook's clear delineation between product and infrastructure teams and Twitter's DevOps-style ownership model, Pete offers valuable comparisons of these contrasting philosophies. The conversation explores: How Dagster provides a higher-level abstraction for data teams, making it easier to track and debug data assets rather than just managing workflows The challenges of modern data platforms and why many organizations struggle with complex, distributed systems that could be simplified A practical approach to migrating from Airflow to Dagster with their "Airlift" toolkit that allows for incremental, low-risk transitions How AI development is fueling demand for better data orchestration as companies build applications that rely on properly managed data pipelines Pete also shares his thoughtful approach to balancing technical debt and product development with a "quarter on, quarter off" cadence that allows teams to both ship features and clean up the inevitable corners that get cut under deadline pressure. For platform engineers, data teams, and technical leaders navigating the intersection of infrastructure and AI, this episode provides practical insights on creating abstractions that deliver real operational value without unnecessary complexity. Guest: Pete Hunt, CEO of Dagster Pete is the CEO of Dagster Labs, where he first joined as Head of Engineering in early 2022 and transitioned into the CEO role later that same year. Before Dagster, Pete co-founded Smyte, an anti-abuse startup acquired by Twitter, where he continued as a senior staff engineer. Earlier in his career, Pete was one of the first engineers to work on Instagram after its acquisition by Facebook in 2012. There, he led development on Instagram’s web and analytics teams and became a co-founder of the React.js project, helping transform an internal experiment into one of the most widely used front-end frameworks in the world. He was also part of the early community around GraphQL and has remained deeply engaged in open source and developer tooling. Pete brings a pragmatic, hands-on perspective to modern data infrastructure. Having been both a founder and an engineer, he focuses on reducing complexity and fatigue in data teams by building tools that actually work together. At Dagster, he remains close to the code and actively involved in technical decisions, combining leadership with deep technical fluency. Pete Hunt, X Dagster Dagster Pipes Dagster Airlift Links to interesting things from this episode: React “Postgres: a Better Message Queue than Kafka?” Airflow Kubeflow CAPES Fargate

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Guests

  • Pete Hunt Guest Inferred

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Taking a Summer Break: What Would You Like To Hear This Fall?

    Jul 15, 20263 minAudio

    We're hitting pause for the summer on new episodes of the Platform Engineering Podcast, but don’t worry, we’ve got some great episodes in the feed to keep you entertained in the meantime. If you’ve got suggestions for...

    • Apple Podcasts

    Credit the people on this episode

  2. What Do Service Meshes Actually Solve? (William Morgan, Buoyant/Linkerd)

    Jun 24, 202656 minAudio

    Network calls fail in ways function calls never do - and once a monolith becomes microservices, reliability problems show up fast: retries amplify load, latency spikes cascade, and “what talks to what?” becomes hard to...

    • Apple Podcasts

    Credit the people on this episode

  3. Continuous Integration at Agentic Velocity with CircleCI’s Rob Zuber

    Jun 10, 202650 minAudio

    When code gets cheaper to produce, feedback becomes the limiting factor - CI, reviews, and the handoffs between tools can quietly slow everything down. Rob Zuber breaks down what platform engineers are seeing as teams...

    • Apple Podcasts

    Credit the people on this episode

  4. Durable Execution for Real‑World Failures with Temporal’s Cornelia Davis

    May 27, 202646 minAudio

    A lot of infrastructure and automation fails for ordinary reasons: rate limits, flaky networks, partial permissions, long-running jobs, and retries that vanish when the process restarts. Durable execution is a way to...

    • Apple Podcasts

    Credit the people on this episode

  5. You Need AI Sysadmins Can Trust, With Cribl's Nikhil Mungel

    May 13, 202655 minAudio

    What happens when a non-deterministic AI system is asked to touch production telemetry or generate changes for an SRE pipeline? The cost of being “close enough” can be lost data, downtime, or a security incident....

    • Apple Podcasts

    Credit the people on this episode