1 episode

The Ins-and-Outs of Data Engineering

The Subsurface Podcast Dremio

    • Technology

The Ins-and-Outs of Data Engineering

    Episode 1: JD Long - The Big Join: Testing pipelines to join 30 billion rows of data… quickly

    Episode 1: JD Long - The Big Join: Testing pipelines to join 30 billion rows of data… quickly

    JD Long is a veteran Quantitative Risk Analyst. He builds stochastic models to predict losses during catastrophic events like hurricanes, earthquakes, or droughts. He shares his data engineering team's painful experience standing up tooling pipelines to load 10s of billions of rows for imbalanced queries into multiple distributed systems. He’s the perfect first guest because he covers multiple tools and techniques and is not shy to share his team's mistakes. He calls it “learning out loud” and I enjoyed every minute of it.

    • 56 min

Top Podcasts In Technology

Acquired
Ben Gilbert and David Rosenthal
Lex Fridman Podcast
Lex Fridman
All-In with Chamath, Jason, Sacks & Friedberg
All-In Podcast, LLC
Lenny's Podcast: Product | Growth | Career
Lenny Rachitsky
Waveform: The MKBHD Podcast
Vox Media Podcast Network
Darknet Diaries
Jack Rhysider