
Is this your show? Claim this page
PodCred
The people behind this show
People get invited back because someone remembered they were great.
PodCred is the verified record of who made this show. A credit here means people who were there confirmed the work.
Hosts
Guests
Who is on this show?
Think of the guest you still think about. The producer who makes the host sound smart. The editor who saves every episode. If a name is missing here, that person is invisible everywhere.
Add a creditEpisodes
521 episodesEntity Resolution at Scale: Graphs, LLMs, and Real-World Data Messiness
Summary In this episode, I spoke with Steven Renwick, CEO and co-founder of Tilores, about the realities of entity-oriented data processing and the practical role of LLMs in entity resolution. We covered the confusing...
How Amazon Uses Telemetry and AI to Automate Physical Logistics
Summary In this episode, I spoke with Keerti Kulkarni about the data and AI systems behind physical logistics automation at Amazon. She explained how her team supports dock-door operations across more than 100 North...
Why Metadata Still Matters: From Data Catalogs to AI-Ready Semantic Hubs
Summary In this episode, I spoke with Christian Bremeau and Simon Dynin about their decades-long work in metadata management and how that experience shaped the evolution of Meta Integration Technology and its...
Reducing Data Debt with Agile Ledger Architecture
Summary In this episode Christopher Doidge talks about his Agile Ledger Architecture (ALA) approach to data warehousing and how it aims to reduce data debt while shortening the path from raw data to trustworthy business...
What Context Really Means in Data Engineering and AI
Summary In this episode Soham Mazumdar, co-founder and CEO of Wisdom.ai, talks about what “context” really means in data engineering and AI systems. He explores why context has become such an overloaded term, spanning...
Specialized AI for Data Engineers: Inside Astronomer’s Otto
Summary In this episode Yetunde Dada discusses Otto, Astronomer’s AI agent for Airflow, and the broader challenge of making agentic tooling actually useful for data engineers. She explored why generic coding assistants...
Why Multi-Agent Systems Need Shared State, Graph Semantics, and Governance
Summary In this episode Ragnor Comerford talks about OmniGraph, a lakehouse-native graph storage layer designed around the needs of agentic systems. He explores how graphs are primarily a semantic model for representing...
Building the Context Flywheel for AI Data Agents
Summary In this episode Prukalpa Sankar, co-founder of Atlan, talks about what it takes to build a “context flywheel” for AI agents in data-intensive organizations. She explained why model intelligence alone isn’t...
Holding Kafka Right: Product-Friendly Streaming with TypeStream
Summary In this episode Jevin Maltais talks about the practical realities of building reliable, product-focused streaming systems with Kafka. Jevin shares lessons from roles at Zapier, Humi, and Clio, where real-time...
Text to Data Products: Kaarvi’s End-to-End AI for Ingestion, Quality, and Dashboards
Summary In this episode Shravan Gunda, founder and CEO of Kaarvi AI, talks about building an AI-native, agent-driven data platform designed to eliminate the janitorial work that consumes most data teams. He explores...
Scaling Graph Analytics Without ETL: Inside PuppyGraph’s Architecture
Summary In this episode Weimo Liu, co‑founder of PuppyGraph, talks about the engineering behind their “zero-copy” graph querying engine for lakehouse and database sources. He explores how PuppyGraph lets you run Cypher...
Maximizing GPU Utilization: Heterogeneous Pipelines with Ray and Kubernetes
Summary In this episode Robert Nishihara, co-founder of Anyscale and co-creator of Ray, talks about maximizing hardware utilization for AI and data-intensive workloads. He explores Ray’s evolution alongside Kubernetes...
The AI-First Data Engineer: 10–50x Productivity and What Changes Next
Summary In this episode, I sit down with Gleb Mezhanskiy, CEO and co-founder of Datafold, to explore how agentic AI is reshaping data engineering. We unpack the leap from chat-assisted coding to truly agentic workflows...
Treat Metering Like Finance: Building Data Platforms for Consumption Economics
Summary In this episode Himant Goyal, Senior Product Manager at Salesforce, talks about how data platform investments enable reliable, accurate metering for consumption-based business models. Himant explains why...
Beyond the PDF: Rowan Cockett on Reproducible, Composable Science
Summary In this episode Rowan Cockett, co-founder and CEO of CurveNote and co-founder of the Continuous Science Foundation, talks about building data systems that make scientific research reproducible, reusable, and...
Beyond Prompts: Practical Paths to Self‑Improving AI
Summary In this episode Raj Shukla, CTO of SymphonyAI, explores what it really takes to build self‑improving AI systems that work in production. Raj unpacks how agentic systems interact with real-world environments, the...
Orion at Gravity: Trustworthy AI Analysts for the Enterprise
Summary In this episode of the Data Engineering Podcast, Lucas Thelosen and Drew Gilson, co-founders of Gravity, discuss their vision for agentic analytics in the enterprise, enabled by semantic layers and broader...
From Models to Momentum: Uniting Architects and Engineers with ER/Studio
Summary In this episode of the Data Engineering Podcast, Jamie Knowles (Product Director) and Ryan Hirsch (Product Marketing Manager) discuss the importance of enterprise data modeling with ER/Studio. They highlight how...
From Data Models to Mind Models: Designing AI Memory at Scale
Summary In this episode of the Data Engineering Podcast, Vasilije "Vas" Markovich, founder of Cognee, discusses building agentic memory, a crucial aspect of artificial intelligence that enables systems to learn, adapt,...
Prompt Management, Tracing, and Evals: The New Table Stakes for GenAI Ops
Summary In this episode of the Data Engineering Podcast, Aman Agarwal, creator of OpenLit, discusses the operational groundwork required to run LLM-powered applications reliably and cost-effectively. He highlights...
From Legacy to AI-Ready: How MongoDB AMP Accelerates Modernization
Summary In this episode, Shilpa Kolhar, SVP of Product and Engineering at MongoDB, discusses using MongoDB as a unified foundation for AI-driven and agentic applications. She explains how the Application Modernization...
Branches, Diffs, and SQL: How Dolt Powers Agentic Workflows
Summary In this episode Tim Sehn, founder and CEO of DoltHub, talks about Dolt - the world’s first version‑controlled SQL database - and why Git‑style semantics belong at the heart of data systems and AI workflows. Tim...
Logical First, Physical Second: A Pragmatic Path to Trusted Data
Summary In this episode of the Data Engineering Podcast Jamie Knowles, Product Director for ER/Studio, talks about data architecture and its importance in driving business meaning. He discusses how data architecture...
Your Data, Your Lake: How Observe Uses Iceberg and Streaming ETL for Observability
Summary In this episode Jacob Leverich, cofounder and CTO of Observe, talks about applying lakehouse architectures to observability workloads. Jacob discusses Observe’s decision to leverage cloud-native warehousing and...
Semantic Operators Meet Dataframes: Building Context for Agents with FENIC
Summary In this episode Kostas Pardalis talks about Fenic - an open-source, PySpark-inspired dataframe engine designed to bring LLM-powered semantics into reliable data engineering workflows. Kostas shares why today’s...
Beyond Dashboards: How Data Teams Earn a Seat at the Table
Summary In this episode Goutham Budati about his Data-Perspective-Action framework and how it empowers data teams to become true business partners. Gautham traces his path from automating Excel reports to leading...
Unfreezing The Data Lake: The Future-Proof File Format
Summary In this episode PhD researcher Xinyu Zeng talks about F3, the “future-proof file format” designed to address today’s hardware realities and evolving workloads. He digs into the limitations of Parquet and ORC -...
From Context to Semantics: How Metadata Powers Agentic AI
Summary In this episode Suresh Srinivas and Sriharsha Chintalapani explore how metadata platforms are evolving from human-centric catalogs into the foundational context layer for AI and agentic systems. They discuss the...
From Data Engineering to AI Engineering: Where the Lines Blur
Summary In this solo episode of the Data Engineering Podcast, host Tobias Macey reflects on how AI has transformed the practice and pace of data engineering over time. Starting from its origins in the Hadoop and cloud...
Malloy: Hierarchical Data, Semantic Models, and the Future of Analytics
Summary In this episode Michael Toy, co-creator of Malloy, talks about rethinking how we work with data beyond SQL. Michael shares the origins of Malloy from his and Lloyd Tabb’s experience at Looker, why SQL’s mental...
Blurring Lines: Data, AI, and the New Playbook for Team Velocity
Summary In this crossover episode, Max Beauchemin explores how multiplayer, multi‑agent engineering is transforming the way individuals and teams build data and AI systems. He digs into the shifting boundary between...
State, Scale, and Signals: Rethinking Orchestration with Durable Execution
Summary In this episode Preeti Somal, EVP of Engineering at Temporal, talks about the durable execution model and how it reshapes the way teams build reliable, stateful systems for data and AI. She explores Temporal’s...
The AI Data Paradox: High Trust in Models, Low Trust in Data
Summary In this episode of the Data Engineering Podcast Ariel Pohoryles, head of product marketing for Boomi's data management offerings, talks about a recent survey of 300 data leaders on how organizations are...
Bridging the AI–Data Gap: Collect, Curate, Serve
Summary In this episode of the Data Engineering Podcast Omri Lifshitz (CTO) and Ido Bronstein (CEO) of Upriver talk about the growing gap between AI's demand for high-quality data and organizations' current data...
Beyond the Perimeter: Practical Patterns for Fine‑Grained Data Access
Summary In this episode of the Data Engineering Podcast Matt Topper, president of UberEther, talks about the complex challenge of identity, credentials, and access control in modern data platforms. With the shift to...
The True Costs of Legacy Systems: Technical Debt, Risk, and Exit Strategies
Summary In this episode Kate Shaw, Senior Product Manager for Data and SLIM at SnapLogic, talks about the hidden and compounding costs of maintaining legacy systems - and practical strategies for modernization. She...
Context Engineering as a Discipline: Building Governed AI Analytics
Summary In this episode of the Data Engineering Podcast, host Tobias Macey welcomes back Nick Schrock, CTO and founder of Dagster Labs, to discuss Compass - a Slack-native, agentic analytics system designed to keep data...
The Data Model That Captures Your Business: Metric Trees Explained
Summary In this episode of the Data Engineering Podcast Vijay Subramanian, founder and CEO of Trace, talks about metric trees - a new approach to data modeling that directly captures a company's business model. Vijay...
From GPUs-as-a-Service to Workloads-as-a-Service: Flex AI’s Path to High-Utilization AI Infra
Summary In this crossover episode of the AI Engineering Podcast, host Tobias Macey interviews Brijesh Tripathi, CEO of Flex AI, about revolutionizing AI engineering by removing DevOps burdens through "workload as a...
From RAG to Relational: How Agentic Patterns Are Reshaping Data Architecture
Summary In this episode of the AI Engineering Podcast Mark Brooker, VP and Distinguished Engineer at AWS, talks about how agentic workflows are transforming database usage and infrastructure design. He discusses the...
Duck Lake: Simplifying the Lakehouse Ecosystem
Summary In this episode of the Data Engineering Podcast Hannes Mühleisen and Mark Raasveldt, the creators of DuckDB, share their work on Duck Lake, a new entrant in the open lakehouse ecosystem. They discuss how Duck...
Aligning Business and Data: The Essential Role of Data Modeling
Summary In this episode of the Data Engineering Podcast Serge Gershkovich, head of product at SQL DBM, talks about the socio-technical aspects of data modeling. Serge shares his background in data modeling and...
From Academia to Industry: Bridging Data Engineering Challenges
Summary In this episode of the Data Engineering Podcast Professor Paul Groth, from the University of Amsterdam, talks about his research on knowledge graphs and data engineering. Paul shares his background in AI and...
High Performance And Low Overhead Graphs With KuzuDB
Summary In this episode of the Data Engineering Podcast Prashanth Rao, an AI engineer at KuzuDB, talks about their embeddable graph database. Prashanth explains how KuzuDB addresses performance shortcomings in existing...
Bridging Data and Decision-Making: AI's Role in Modern Analytics
Summary In this episode of the Data Engineering Podcast Lucas Thelosen and Drew Gilson from Gravity talk about their development of Orion, an autonomous data analyst that bridges the gap between data availability and...
From Bits to Tables: The Evolution of S3 Storage
Summary In this episode of the Data Engineering Podcast Andy Warfield talks about the innovative functionalities of S3 Tables and Vectors and their integration into modern data stacks. Andy shares his journey through...
Revolutionizing Python Notebooks with Marimo
Summary In this episode of the Data Engineering Podcast Akshay Agrawal from Marimo discusses the innovative new Python notebook environment, which offers a reactive execution model, full Python integration, and built-in...
Warehouse Native Incremental Data Processing With Dynamic Tables And Delayed View Semantics
Summary In this episode of the Data Engineering Podcast Dan Sotolongo from Snowflake talks about the complexities of incremental data processing in warehouse environments. Dan discusses the challenges of handling...
Streamlining Data Pipelines with MCP Servers and Vector Engines
Summary In this episode of the Data Engineering Podcast Kacper Łukawski from Qdrant about integrating MCP servers with vector databases to process unstructured data. Kacper shares his experience in data engineering,...
Foundational Data Engineering At Two Sigma
Summary In this episode of the Data Engineering Podcast Effie Baram, a leader in foundational data engineering at Two Sigma, talks about the complexities and innovations in data engineering within the finance sector....
Enabling Agents In The Enterprise With A Platform Approach
Summary In this episode of the Data Engineering Podcast Arun Joseph talks about developing and implementing agent platforms to empower businesses with agentic capabilities. From leading AI engineering at Deutsche...
Dagster's New Era: Modularizing Data Transformation in the Age of AI
Summary In this episode of the Data Engineering Podcast we welcome back Nick Schrock, CTO and founder of Dagster Labs, to discuss the evolving landscape of data engineering in the age of AI. As AI begins to impact data...
AI and the Lakehouse: How Starburst is Pioneering New Workflows
Summary In this episode of the Data Engineering Podcast Alex Albu, tech lead for AI initiatives at Starburst, talks about integrating AI workloads with the lakehouse architecture. From his software engineering roots to...
Amazon S3: The Backbone of Modern Data Systems
Summary In this episode of the Data Engineering Podcast Mai-Lan Tomsen Bukovec, Vice President of Technology at AWS, talks about the evolution of Amazon S3 and its profound impact on data architecture. From her work on...
Scaling Data Operations With Platform Engineering
Summary In this episode of the Data Engineering Podcast Chakravarthy Kotaru talks about scaling data operations through standardized platform offerings. From his roots as an Oracle developer to leading the data platform...
From Data Discovery to AI: The Evolution of Semantic Layers
Summary In this episode of the Data Engineering Podcast, host Tobias Macy welcomes back Shinji Kim to discuss the evolving role of semantic layers in the era of AI. As they explore the challenges of managing vast data...
Balancing Off-the-Shelf and Custom Solutions in Data Engineering
Summary In this episode of the Data Engineering Podcast Tulika Bhatt, a senior software engineer at Netflix, talks about her experiences with large-scale data processing and the future of data engineering technologies....
StarRocks: Bridging Lakehouse and OLAP for High-Performance Analytics
Summary In this episode of the Data Engineering Podcast Sida Shen, product manager at CelerData, talks about StarRocks, a high-performance analytical database. Sida discusses the inception of StarRocks, which was forked...
Exploring NATS: A Multi-Paradigm Connectivity Layer for Distributed Applications
Summary In this episode of the Data Engineering Podcast Derek Collison, creator of NATS and CEO of Synadia, talks about the evolution and capabilities of NATS as a multi-paradigm connectivity layer for distributed...
Advanced Lakehouse Management With The LakeKeeper Iceberg REST Catalog
Summary In this episode of the Data Engineering Podcast Viktor Kessler, co-founder of Vakmo, talks about the architectural patterns in the lake house enabled by a fast and feature-rich Iceberg catalog. Viktor shares his...
Simplifying Data Pipelines with Durable Execution
Summary In this episode of the Data Engineering Podcast Jeremy Edberg, CEO of DBOS, about durable execution and its impact on designing and implementing business logic for data systems. Jeremy explains how DBOS's...
Overcoming Redis Limitations: The Dragonfly DB Approach
Summary In this episode of the Data Engineering Podcast Roman Gershman, CTO and founder of Dragonfly DB, explores the development and impact of high-speed in-memory databases. Roman shares his experience creating a more...
Bringing AI Into The Inner Loop of Data Engineering With Ascend
Summary In this episode of the Data Engineering Podcast Sean Knapp, CEO of Ascend.io, explores the intersection of AI and data engineering. He discusses the evolution of data engineering and the role of AI in automating...
Astronomer's Role in the Airflow Ecosystem: A Deep Dive with Pete DeJoy
Summary In this episode of the Data Engineering Podcast Pete DeJoy, co-founder and product lead at Astronomer, talks about building and managing Airflow pipelines on Astronomer and the upcoming improvements in Airflow...
Accelerated Computing in Modern Data Centers With Datapelago
Summary In this episode of the Data Engineering Podcast Rajan Goyal, CEO and co-founder of Datapelago, talks about improving efficiencies in data processing by reimagining system architecture. Rajan explains the shift...
The Future of Data Engineering: AI, LLMs, and Automation
Summary In this episode of the Data Engineering Podcast Gleb Mezhanskiy, CEO and co-founder of DataFold, talks about the intersection of AI and data engineering. He discusses the challenges and opportunities of...
Evolving Responsibilities in AI Data Management
Summary In this episode of the Data Engineering Podcast Bartosz Mikulski talks about preparing data for AI applications. Bartosz shares his journey from data engineering to MLOps and emphasizes the importance of data...
CSVs Will Never Die And OneSchema Is Counting On It
Summary In this episode of the Data Engineering Podcast Andrew Luo, CEO of OneSchema, talks about handling CSV data in business operations. Andrew shares his background in data engineering and CRM migration, which led...
Breaking Down Data Silos: AI and ML in Master Data Management
Summary In this episode of the Data Engineering Podcast Dan Bruckner, co-founder and CTO of Tamr, talks about the application of machine learning (ML) and artificial intelligence (AI) in master data management (MDM)....
Building a Data Vision Board: A Guide to Strategic Planning
Summary In this episode of the Data Engineering Podcast Lior Barak shares his insights on developing a three-year strategic vision for data management. He discusses the importance of having a strategic plan for data,...
How Orchestration Impacts Data Platform Architecture
Summary The core task of data engineering is managing the flows of data through an organization. In order to ensure those flows are executing on schedule and without error is the role of the data orchestrator. Which...
An Exploration Of The Impediments To Reusable Data Pipelines
Summary In this episode of the Data Engineering Podcast the inimitable Max Beauchemin talks about reusability in data pipelines. The conversation explores the "write everything twice" problem, where similar pipelines...
The Art of Database Selection and Evolution
Summary In this episode of the Data Engineering Podcast Sam Kleinman talks about the pivotal role of databases in software engineering. Sam shares his journey into the world of data and discusses the complexities of...
Bridging Code and UI in Data Orchestration with Kestra
Summary In this episode of the Data Engineering Podcast, Anna Geller talks about the integration of code and UI-driven interfaces for data orchestration. Anna defines data orchestration as automating the coordination of...
Streaming Data Into The Lakehouse With Iceberg And Trino At Going
In this episode, I had the pleasure of speaking with Ken Pickering, VP of Engineering at Going, about the intricacies of streaming data into a Trino and Iceberg lakehouse. Ken shared his journey from product engineering...
An Opinionated Look At End-to-end Code Only Analytical Workflows With Bruin
Summary The challenges of integrating all of the tools in the modern data stack has led to a new generation of tools that focus on a fully integrated workflow. At the same time, there have been many approaches to how...
Feldera: Bridging Batch and Streaming with Incremental Computation
Summary In this episode of the Data Engineering Podcast, the creators of Feldera talk about their incremental compute engine designed for continuous computation of data, machine learning, and AI workloads. The...
Accelerate Migration Of Your Data Warehouse with Datafold's AI Powered Migration Agent
Summary Gleb Mezhanskiy, CEO and co-founder of DataFold, joins Tobias Macey to discuss the challenges and innovations in data migrations. Gleb shares his experiences building and scaling data platforms at companies like...
Bring Vector Search And Storage To The Data Lake With Lance
Summary The rapid growth of generative AI applications has prompted a surge of investment in vector databases. While there are numerous engines available now, Lance is designed to integrate with data lake and lakehouse...
The Role of Python in Shaping the Future of Data Platforms with DLT
Summary In this episode of the Data Engineering Podcast, Adrian Broderieux and Marcin Rudolph, co-founders of DLT Hub, delve into the principles guiding DLT's development, emphasizing its role as a library rather than a...
Posts
No posts yet. Posts Data Engineering Podcast shares on PodSubs, and what it publishes on Substack, YouTube and its own site, will show here.

Subscribe
Be on the front line of the show
Everything Data Engineering Podcast sends its own people, first. Resources, announcements, behind the scenes, events, and giveaways, straight from the host. No account needed.
- The resources the host shares with subscribers
- Announcements before anyone else hears
- Behind the scenes from the team
- Events, invitations, and giveaways
- Only what the host sends, nothing else
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.














