The Databricks Data Engineer

Jakub Lasak

Helping 18k+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors.

  1. לפני 3 ימים

    The real reason your Databricks week never got simpler - it isn't the lakehouse

    Your org finished the migration. Two systems became one. The nightly copy nobody wanted to own is gone, and so is the quarterly meeting where finance and the data science team each showed up with a real revenue number from tables in different buildings. That pain was real, and it is genuinely dead. So why does your week feel heavier than the deck promised? Because most of what got sold as simplification was a relocation. The copy didn't die, it turned into an argument about which schema in which catalog is the real one. The sync job didn't die, it became a maintenance calendar that pages nobody as an alert and pages you months later as the job that got a little slower every week. In this episode: - Why every architecture in this field gets sold on removal, and what the slide never names - How to tell the work that genuinely died in the lakehouse from the work that only changed address - Why a heavier week after a clean migration is not a verdict on you - The two questions to run on the next architecture pitched to your org, and why the second one is about names, not tasks This episode is for Databricks data engineers who got the win the lakehouse promised and still can't explain why the calendar filled back up. Whether you ran the migration or inherited it, you'll walk away with a lens you can run on any architecture sold as the thing that finally makes data simple. --- Helping 18,000+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors. Follow The Databricks Data Engineer for new episodes every week. LinkedIn: linkedin.com/in/jrlasak Newsletter: dataengineer.wiki #DataEngineering #Databricks #DataEngineer #CareerGrowth #ApacheSpark #DeltaLake

    The real reason your Databricks week never got simpler - it isn't the lakehouse

  2. 31 באוג׳

    The Databricks incident where the most senior engineer never touched the keyboard

    Thursday, just after two in the afternoon. The first message doesn't come from an alert, it comes from a person in revenue operations: the biggest region jumped overnight and nobody sold anything new. Nobody answers. The engineer who owns the nightly Lakeflow job finds it in the run history. An overnight failure, a restart before anyone was awake, a window of orders written a second time under every executive dashboard in the company. So the fixing starts immediately. Head down, no message, because typing in the channel feels like stealing time from the fix. Which is how a second engineer joins a silent channel, starts the same correction, and lands the same window twice again. Effort was never the scarce thing in that channel. In this episode: - Why every outage is really two problems running in parallel, and only one of them has an owner - What a staff engineer did in ninety seconds without opening a notebook, and why the slower fix was the point - How to state an impact boundary a non engineer can repeat, including the bucket everyone skips - The one sentence that takes the most underpriced role in an incident, at any level This episode is for Databricks data engineers who have sat in a chaotic incident channel watching questions pile up faster than anyone can answer them. Whether you own the broken job or you're wondering if it's your place to speak, you'll know which seat in that channel is empty and how to take it. --- Helping 18,000+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors. Follow The Databricks Data Engineer for new episodes every week. LinkedIn: linkedin.com/in/jrlasak Newsletter: dataengineer.wiki #DataEngineering #Databricks #DataEngineer #CareerGrowth #ApacheSpark #DeltaLake

    The Databricks incident where the most senior engineer never touched the keyboard

  3. 24 באוג׳

    9 Databricks decisions that reveal your real seniority (and one that only looks senior)

    Three in the morning. The nightly job that loads the orders table died halfway. Half the tasks are green, and the rerun button is right there. What you do in the next ten seconds says more about your level than anything on your resume. Nobody places you by your title or your years. They place you by a pattern of routine calls made fast, in the dark, and everyone around you can read it except you. In this episode: - The ten-second question to ask before you rerun anything that writes - Why the fastest fix for a slow Databricks job is often the one that renews its cost every month - How the flaky job you quietly restart every week stops being a defect and starts being weather - What it means that most data engineers cannot price their own job, and what changes when they can - The one call that reads as the most senior contribution in the room and almost never is This episode is for Databricks data engineers who have the title, or want it, and want to know how hiring managers actually place them. Whether you are prepping for a senior interview or wondering why your level has not moved in two review cycles, you will walk away with a lens for your very next decision. --- Helping 18,000+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors. Follow The Databricks Data Engineer for new episodes every Monday, Wednesday, and Friday. LinkedIn: linkedin.com/in/jrlasak Newsletter: dataengineer.wiki #DataEngineering #Databricks #DataEngineer #CareerGrowth #ApacheSpark #DeltaLake

    9 Databricks decisions that reveal your real seniority (and one that only looks senior)

  4. 17 באוג׳

    The Databricks job market in 2026: there are two of them, and only one answers

    One engineer tells you hiring is back and recruiters are warm again. The next one is sixty applications deep and hearing nothing. Same month, same platform, sometimes the same city. That sounds like noise. It isn't. It's a split, and what sorts you onto one side of it has almost nothing to do with how good you are. In this episode: - Why forty applications in a week returns two dead screens, while a smaller and pickier search gets answered - The three things that changed in the last year and turned the generic Databricks data engineer posting into a stack of a hundred before lunch - Why fewer postings can be the better market for you, and the one condition that has to hold for it - The honest case against specializing, plus the test for which second surface is actually safe to bet on - The one resume sentence to rewrite this week, using work you already did and never wrote down This episode is for Databricks data engineers hearing two contradictory versions of the 2026 market and unsure which one is theirs. Whether you're mid-level and stuck in the application pile, senior and getting less traction than your years should buy, or early career and facing the most crowded entry market in a while, you'll walk away knowing which market your resume is sitting in and what it costs to move. --- Helping 18,000+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors. Follow The Databricks Data Engineer for new episodes every Monday, Wednesday, and Friday. LinkedIn: linkedin.com/in/jrlasak Newsletter: dataengineer.wiki #DataEngineering #Databricks #DataEngineer #CareerGrowth #ApacheSpark #DeltaLake

    The Databricks job market in 2026: there are two of them, and only one answers

  5. 10 באוג׳

    Databricks broadcast joins: when the memo beats the meeting (and when it kills a task)

    Two engineers on the same team join the same big orders table to the same small lookup table. Same cluster, same data, one line of code different. Sarah's finishes in the time it takes to get a coffee. Mike's dies, and the error isn't about the data at all. A task ran out of memory building a hash map. Broadcast joins get passed around as a tip instead of a model: small table equals fast, flip the switch when a join runs long. But the mechanism in almost every write-up is years out of date, and the size Spark checks is not the size that has to fit in memory. In this episode: - How to explain broadcast versus shuffle joins at standup, in one sentence, without a whiteboard - What actually happens on a current Databricks runtime when a join broadcasts, and why the popular explanation stopped being true - Why the size estimate Spark trusts is not the size that lands, and where to read the real payload - What adaptive query execution rescues you from, and the two places its hands are tied - The question to ask before you add a broadcast hint, and the failure signature that tells you a broadcast is what broke This episode is for Databricks data engineers who write joins every week and treat the broadcast hint as a speed switch. Whether you're mid-level and tired of guessing why one join flies and an identical one falls over, or senior and about to be asked in an interview why Spark chose a sort merge join, you'll walk away able to predict the call before you run the query. --- Helping 18,000+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors. Follow The Databricks Data Engineer for new episodes every week. LinkedIn: linkedin.com/in/jrlasak Newsletter: dataengineer.wiki #DataEngineering #Databricks #DataEngineer #CareerGrowth #ApacheSpark #DeltaLake1

    Databricks broadcast joins: when the memo beats the meeting (and when it kills a task)

  6. 4 באוג׳

    Build the platform or use the platform: the two Databricks data engineer tracks nobody names

    Two Databricks engineers sit at adjacent desks. Same title, same pay, same words in the ladder document. Watch a year go by, and they are not doing the same job. Both get rated strong. Both get told they're ready for more scope. Nobody says the useful thing: the evidence those two are stacking isn't interchangeable, and one of those years makes no sense on the other one's promotion packet. In this episode: - Why the data engineering ladder quietly forks, and why that fork is missing from every ladder document - The five-minute question that tells you which track your last year of Databricks work actually built toward - How each track reaches staff, and the very different proof each demands - Where each track stalls a good year, and what to say in your next one on one to get moved off it This episode is for Databricks data engineers past the junior stage who keep getting strong reviews and vague answers about what's next. Whether you write the cluster policies and ingestion frameworks or ship the tables the business argues from, you'll walk away able to name your track and ask for the work that gets you to the next rung. --- Helping 18,000+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors. Follow The Databricks Data Engineer for new episodes every Monday, Wednesday, and Friday. LinkedIn: linkedin.com/in/jrlasak Newsletter: dataengineer.wiki #DataEngineering #Databricks #DataEngineer #CareerGrowth #ApacheSpark #DeltaLake

    Build the platform or use the platform: the two Databricks data engineer tracks nobody names

  7. 27 ביולי

    Why Unity Catalog exists: the Databricks governance chaos it was built to end

    You inherited Unity Catalog already switched on. You learned the catalogs, the schemas, the grant statements, and never once saw the problem all of it was built to solve. Then someone in a review asks why the company suffered through that migration, and the best you've got is one word: governance. Here's what that word hides. Before Unity Catalog, one company ran five separate Databricks workspaces, each walled off - its own tables, its own users, its own rules, and nothing connecting them. A new analyst couldn't find where a table lived without three Slack threads. An auditor's one question - who touched customer data last quarter - cost two weeks of digging through old notebooks. In this episode: - Why every Unity Catalog design choice is scar tissue over a specific pain from the workspace-per-team era - How to explain what the migration actually bought your company in sixty seconds, to a product manager or a staff engineer - The difference between the engineer who says "governance" and the one who gets handed the next platform decision - Why turning Unity Catalog on does not automatically give you what it promises, and what actually does This episode is for Databricks data engineers who use Unity Catalog every day but couldn't explain to a product manager why it exists. Whether you joined after the migration and only ever saw the dropdowns, or you're heading into a design conversation where you'll need to justify the work, you'll walk away able to name the world that came before and read every feature as an answer to it. --- Helping 18,000+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors. Follow The Databricks Data Engineer for new episodes every Monday, Wednesday, and Friday. LinkedIn: linkedin.com/in/jrlasak Newsletter: dataengineer.wiki #DataEngineering #Databricks #DataEngineer #CareerGrowth #ApacheSpark #DeltaLake

    Why Unity Catalog exists: the Databricks governance chaos it was built to end

  8. 20 ביולי

    How to get credit for Databricks platform work without becoming the on-call martyr

    Your pipeline hasn't failed in months, and nobody noticed. Then it breaks at 2am, you fix it in twenty minutes, and Slack fills with fire emojis and thank-yous. You're the backbone of the team. You've also been at the same level for three years, and nobody can quite explain why. It's not bad luck, and it's not a skill gap. The exact thing everyone praises you for, being the one who can always fix it, is the reason they can't afford to move you up. In this episode: - Why being the indispensable on-call hero quietly caps the level you can reach - How one engineer systematized a whole class of Databricks incidents out of existence and got promoted for it - The difference between measuring your work in saves and measuring it in risk retired - How to make yourself removable from the hero seat without handing away your job security - The one self-test that tells you whether your recognition depends on things staying broken This episode is for Databricks data engineers who carry the pager and keep the platform standing, but keep watching less essential peers get promoted first. Whether you're the war-room legend or the quiet engineer whose prevention work goes uncredited, you'll walk away with three concrete moves to turn indispensability into a promotion case. --- Helping 18,000+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors. Follow The Databricks Data Engineer for new episodes every Monday, Wednesday, and Friday. LinkedIn: linkedin.com/in/jrlasak Newsletter: dataengineer.wiki #DataEngineering #Databricks #DataEngineer #CareerGrowth #ApacheSpark #DeltaLake

    How to get credit for Databricks platform work without becoming the on-call martyr
הצגת הכול (24)

אודות

Helping 18k+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors.

מידע

  • יוצרים
    Jakub Lasak
  • שנות פעילות
    2026
  • פרקים
    24
  • סיווג
    תוכן הולם
  • זכויות יוצרים
    © Jakub Lasak
  • אתר התכנית
    The Databricks Data Engineer

אולי יעניין אותך