Data Engineering Central Podcast

Data Engineering in Real Life

Long Live the Data Engineer. No holds barred. Talking about Data Engineering news, topics, and general mayhem. dataengineeringcentral.substack.com

  1. The Evolution of Software, Streaming, and Data Engineering with Robin Moffatt

    HACE 6 H

    The Evolution of Software, Streaming, and Data Engineering with Robin Moffatt

    In this episode, I sit down with industry veteran Robin Moffatt — Sr. Principal Advisor in Streaming Data Technologies (Kafka, etc.) and a longtime voice in the data engineering community, to unpack the journey from old-school data architectures to today’s real-time streaming ecosystems. From early mainframe data processing and COBOL through the rise of Apache Kafka, streaming ETL, and event-driven systems, Robin shares lived experience from across decades of building, scaling, and evolving data platforms. We dive into: * 🧠 How the role of software engineering has shifted with the rise of distributed, real-time systems * 📊 Why event streaming and platforms like Kafka aren’t just messaging systems, but the backbone of modern data architectures * 🚀 How the community’s tooling and mental models have had to evolve — from static databases and nightly jobs to continuous, always-on streaming applications * 🤖 A candid look at how AI and real-time data are intersecting, shaping both tooling and expectations for the next decade * 🔮 Robin’s perspective on where the industry is headed — beyond buzzwords toward real engineering maturity Along the way, we get historical context, real-world lessons from conference stages and community forums, and a perspective on building resilient, scalable systems that power today’s data-rich applications. If you’ve ever wondered how we got from batch jobs to continuous event streams, or what it really takes to build modern pipelines that support AI workflows, this conversation with Robin is a must-listen. For more from Robin: * 📍 His personal blog & talks: https://rmoff.net/ * 🔗 LinkedIn profile: https://www.linkedin.com/in/robinmoffatt Thanks for reading Data Engineering Central! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

    50 min

  2. HACE 6 DÍAS

    The Lakehouse Architecture: Multimodal Data, Delta Lake, and the Future of Data Engineering (with R. Tyler Croy)

    In this episode of the Data Engineering Central Podcast, I sit down with R. Tyler Croy for a wide-ranging conversation on the present—and future—of modern data platforms. Tyler is a long-time open-source contributor to projects such as delta-rs. You can watch him on YouTube, read his blog, or work directly with him through his consultancy, Buoyant Data. Tyler has spent years deep in the open-source data ecosystem, contributing to projects such as Delta Lake and thinking critically about how real-world data systems are built and maintained. This isn’t a hype-driven conversation—it’s a grounded discussion about what’s working, what’s breaking, and what’s coming next. We dig into: * What the Lakehouse architecture gets right—and where it still falls short * Why multimodal data (text, images, audio, video, embeddings) changes everything * How open table formats like Delta Lake fit into the next generation of data platforms * The growing gap between data tooling hype and day-to-day data engineering reality * What skills and architectural thinking will matter most for data engineers over the next decade If you’re building or operating modern data platforms—and trying to separate real signal from noise—this episode is for you. Thanks for reading Data Engineering Central! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

    59 min

  3. 28 ENE

    Building the Full Data Stack and the Audience That Comes With It

    In this episode of the Data Engineering Central Podcast, I sit down with Hoyt Emerson, founder of The Full Data Stack and Early Signal, for a wide-ranging conversation on data, analytics, and creating content in the tech world. We talk candidly about: * What actually matters in modern data and analytics * Why so much “data content” misses the mark * The difference between noise and real signal * What works (and doesn’t) when building a technical audience * Writing, consistency, and credibility in the data space * Why opinions + experience beat trends and buzzwords If you’re a data engineer, analyst, or technologist who’s curious about both building better data systems and communicating ideas that resonate, this episode goes deep on the lessons learned from doing both. This is less about hacks—and more about craft, judgment, and long-term thinking. Thanks for reading Data Engineering Central! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

    46 min

  4. 20 ENE

    From Wiring Circuits to Data Pipelines

    In this episode of the Data Engineering Central Podcast, I sit down with Andy Leonard — someone who’s been building systems long before “data engineering” was even a job title. Andy’s career didn’t start in software at all. It started with physical circuits, literally wiring systems as an electrician, before moving into programming, databases, and eventually decades of hands-on data engineering work. This conversation isn’t about trends or hype cycles. It’s about how the fundamentals of data work have evolved, what hasn’t changed, and what you only learn after years of building, breaking, fixing, and rebuilding real systems. We talk about how the industry got here, how tools have changed, where they haven’t helped as much as advertised, and what newer data engineers can learn from a long, practical career spent close to the metal. If you’re interested in perspective, experience, and lessons earned the hard way — this one’s for you. Thanks for reading Data Engineering Central! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

    2 h y 10 min
  5. From DBA to Data Everything

    14 ENE

    From DBA to Data Everything

    In this episode of the Data Engineering Central Podcast, I interview a Data OG, someone who’s been around the data space forever, and we talked about all things data, past, present, and future. I’m joined by Thomas Horton a longtime friend and one of the most well-rounded data professionals I know. Over the course of his career, Tom has worn just about every hat in data: developer, DBA, analyst, and everything in between. He’s lived through the era of on-prem databases, the rise of analytics, and the constant reinvention that defines modern data engineering today. We talk about what’s changed, what hasn’t, and why many of the “new” problems in data feel oddly familiar. We also dig into lessons learned the hard way, lessons that are just as relevant for early-career data engineers as they are for seasoned practitioners navigating today’s ever-expanding stacks. On a personal note, a huge portion of what I know about relational databases and analytics can be traced back to Tom. This conversation is part reflection, part history lesson, and part reality check on where the data industry is headed next. * If you’re interested in the past, present, and future of data—and what really matters beneath all the tooling, this is an episode you won’t want to miss. Thanks for reading Data Engineering Central! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

    1 h y 6 min

  6. 17/12/2025

    Scott Haines on the Future of Data Engineering

    In this episode, I sit down with Scott Haines — O’Reilly author, Databricks MVP, and veteran of Yahoo, Nike, and Twilio — for a wide-ranging conversation on the real state of modern data engineering. We dig into open-source ecosystems, Lakehouse architectures, the evolution of Spark, streaming, what’s broken and what’s working in today’s data tooling, and the lessons Scott has learned scaling platforms at some of the biggest companies in the world. If you care about data engineering, architecture, OSS, or the future of the modern data stack, you’ll love this one. Thanks for reading Data Engineering Central! This post is public so feel free to share it. Make sure to follow Scott here on Substack, and over on GitHub. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

    1 h y 51 min

  7. 13/11/2025

    Data Engineering Central Podcast - 09

    Hello! A new episode of the Data Engineering Central Podcast is dropping today. We will be covering a few hot topics! * Cluster Fatigue * The Death of Open Source Going to be a great show, come along for the ride! Thanks for reading Data Engineering Central! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit dataengineeringcentral.substack.com/subscribe

    7 min

  8. 10/07/2025

    Data Engineering Central Podcast - Episode 8

    This is a free preview of a paid episode. To hear more, visit dataengineeringcentral.substack.com Hello! A new episode of the Data Engineering Central Podcast is dropping today, we will be covering a few hot topics! * Apache Iceberg Catalogs * new Boring Catalog * new full Iceberg support from Databricks/Unity Catalog * Databricks SQL Scripting * DuckDB coming to a Lake House near you * Lakebase from Databricks Going to be a great show, come along for the ride! Thanks …

    6 min
Ver todo (16)

Acerca de

Long Live the Data Engineer. No holds barred. Talking about Data Engineering news, topics, and general mayhem. dataengineeringcentral.substack.com

Información

  • Creador
    Data Engineering in Real Life
  • Años de actividad
    2024 - 2026
  • Episodios
    16
  • Clasificación
    Explícito
  • Copyright
    © dataengineeringdude
  • Mostrar sitio web
    Data Engineering Central Podcast

También te podría interesar