DOP 333: The Hidden Problems Behind Every Data Pipeline

: Pete Hunt, CEO of Dagster and early React team member, explores the evolution from Facebook's early React development through trust and safety infrastructure at Twitter, to building modern data orchestration tools. The conversation reveals how similar infrastructure problems plague every industry - whether you're launching rockets or managing porta-potties, the core challenges remain consistent: late data, quality issues, and mysterious errors that require both automated solutions and human oversight. The discussion dives into the technical realities of scaling systems, from the microservices complexity trap to the current AI adoption wave. Hunt shares candid insights about leadership challenges, including how well-intentioned technology recommendations can backfire, and why most data projects fail despite sophisticated multi-agent orchestration. The conversation touches on career advancement pressures that drive unnecessary complexity and the importance of focusing on actual user adoption rather than technical sophistication. This episode features Pete Hunt in conversation with hosts Darin and Viktor, covering everything from regular expression nightmares to the future of data infrastructure and the lessons learned from building products that people actually use. Pete's contact information: X: LinkedIn: YouTube channel: Review the podcast on Apple Podcasts: Slack: Connect with us at:

You can listen to DOP 333: The Hidden Problems Behind Every Data Pipeline online on Radio and Podcast. Open the player on this page to stream the available audio.

DOP 333: The Hidden Problems Behind Every Data Pipeline is an episode from DevOps Paradox by Darin Pope.

This episode is 51:11 long.

This episode was published on Jan 14, 2026.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from DevOps Paradox when more episodes are available from the podcast feed.