Data Gibberish

Yordan Ivanov
  • 0.0 (0)
  • TECHNOLOGY

Data Gibberish helps experienced data engineers and data leads handle stakeholder requests, scope pressure, and decision-making with scripts, checklists, and playbooks you can use immediately. www.datagibberish.com

Episodes

  1. 3D AGO

    How to Transition from Managing Pipelines to Managing People like Prajakta

    Presenting Prajakta Yerpude Prajakta spent a decade climbing every rung of the data engineering ladder, intern, engineer, senior, lead, manager, and somewhere along the way realized her biggest impact was no longer in the code she wrote but in the people she unblocked. Getting promoted into management felt like winning. I had the title, the 1:1s, the org chart with my name at the top. What nobody told me was that everything I had spent years getting good at had just become irrelevant. The skills that got me the promotion were the wrong ones for the job. Speed, ownership, technical control, those are IC superpowers. In management, they become liabilities. Prajakta figured this out the hard way too, after a decade climbing every rung of the data engineering ladder. What she learned on the other side is clearer than anything I've read on the topic, and it maps almost exactly to where most senior data engineers quietly stall You Are Already a Manager Before the Title Most people wait for the promotion to start operating differently. That is the wrong order. The data professionals who get promoted into leadership are not the ones who wanted the title. They are the ones who were already doing the work before anyone gave it to them. Clarifying requirements with stakeholders when nobody asked. Helping teammates get unblocked when they had their own deliverables to ship. Asking the why behind the problem instead of just solving the what. The Week Your Code Sits Untouched There is a specific moment that signals the shift. You get to the end of a week and realize you made almost no progress on your own work — but the team moved forward because of the conversations you had. That discomfort is the job changing underneath you. If you wait until the title to start operating at the next level, you are asking your manager to take a bet on future behavior. That is a harder sell than showing them behavior that is already there. Speaking Up Is a Skill Early in any leadership journey, the most uncomfortable moment is the same for almost everyone: being in a room of more experienced people, knowing something is wrong, and deciding whether to say it. The imposter syndrome is real. The difference is learning not to let it make the decision for you. Facts Remove the Politics In data, you have a specific advantage: you can ground a challenge in numbers and evidence. When you do that, you shift the conversation from opinion to information. The most experienced person in the room does not win by default anymore. Leadership is about stepping up when something needs to be said. That is available to you at any level. An intern can flag a bad assumption. A senior IC can challenge a decision made three levels above them. The cost of staying quiet is harder to measure than the cost of speaking up. But it compounds over time. Empathy Is Leverage. As engineers, we are trained to solve technical problems. The higher you go, the more you realize the hardest problems are people problems. Misalignment. Stress. Lack of clarity. Someone having an off day. These do not have a clean solution. Some of them do not need a solution at all. Instead, they need listening. The Problem-Solver Trap The instinct to fix things immediately is exactly what gets senior ICs into leadership. It is also the thing that makes early management hard. Not every situation calls for a solution. Sometimes the person in front of you just needs to feel heard. When you learn to read that distinction — when you can tell the difference between a problem that needs solving and a person who needs to know you are paying attention — the team responds differently. People who feel understood and supported perform better without you pushing harder. You do not manufacture that with a process or a framework. It is built through consistent, genuine attention to what is actually going on beneath the surface. Technical skills get you to the role. Empathy and emotional intelligence determine how far you go inside it. Your Impact Becomes Invisible. That Is the Whole Point. As an IC, your output is visible. Code you wrote. Pipelines you fixed. A cost number you moved. You can point to it directly. As a manager, your impact shows up through other people. Better decisions. Stronger execution. A team delivering consistently and growing in the process. None of that has your name on it. Unlearning the Need to Own the Outcome The hardest thing to unlearn is tying your personal value to what you directly produce. Andy Grove put it clearly in High Output Management: your output as a manager is your team’s output. That is the measure. You do not have a separate output anymore. The team speaks on your behalf. That means impact can look like: * Creating the right direction when things are ambiguous * Removing friction before it slows the team down * Helping someone succeed in a way they would not have without your support * Building the environment in which good work happens consistently You stop building systems. You start building the conditions in which good systems get built. That is more scalable than anything you could ship yourself. The Myth That Needs to Die The most persistent management myth: a good manager always has to be in control of everything. New managers fall into it. Experienced managers fall into it. They believe that effectiveness means knowing every detail, monitoring closely, and constantly stepping in. What that actually creates is dependency. It slows people down. It makes you a single point of failure. The Real Measure The actual measure of good leadership is when your team can collaborate, problem-solve, and resolve issues without you needing to be in the room. When something gets fixed before you even know it happened. That is the job done well. High standards work better when trust already exists. When people know you genuinely support them, they are more open to feedback, more willing to stretch, more willing to take on work that makes them uncomfortable. You can be kind and still be clear. You can be supportive and still hold people accountable. The balance is a more effective version of both. Final Thoughts The move from IC to manager is a step sideways into a completely different job that happens to sit inside the same organization. The data professionals who stall at senior are almost always technically strong. They can build. What they have not yet built is the capacity to make others better at building. If you are waiting until you feel ready to step into that, here is the honest version: you will never feel fully ready. Growth happens because you stepped forward before you did. — Yordan More on the Topic These are some of the articles Prajakta mentioned in our chat: * The Operating System Every Data Engineering Leader Needs * Business value mapping playbook: How to translate data work into executive language * How To Stop Waiting For Permission To Lead Data Engineering * Why Senior Data Engineers Lose Their Velocity (Even When Their Skills Improve) This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.datagibberish.com/subscribe

    51 min

  2. APR 10

    University Doesn't Teach Data Engineering. Here's What You're Missing.

    I had a chat with Mihail, a software engineering lead who also lectures at Plovdiv University. He teaches data engineering as an elective because the core CS curriculum has zero room for it. That tells you everything about the state of data education. Universities produce graduates who know SQL syntax, basic normalization, and enough theory to pass an exam. Then those graduates walk into jobs where the theory is the easy part. The pipelines, the warehouses, the cloud platforms, the stakeholder conversations, the ambiguity of real problems. None of that shows up in a lecture hall. I got my databases grade in university and it was mediocre. I didn’t care because the course felt abstract and disconnected from anything real. Most students feel the same way. And the system does nothing to change that. The gap between what university teaches and what the job demands keeps growing. But the fix is simpler than most people think, and it starts long before graduation day. The Curriculum Stops at SQL University data education peaks at a single databases course in your second year. You learn what SQL is, what a database does, and some theory about normalization and data storage. The course is abstract. Students treat databases like glorified Excel files because nobody shows them why the syntactical overhead matters. Until you work on large-scale projects, the difference between a spreadsheet and a relational database feels like a bureaucratic formality. And that databases course is the baseline for everything data-related in the entire degree. Everything Beyond SQL Is “Too Specialized” From that point, data science branches into math and statistics. Data engineering branches into pipelines, warehouses, cloud platforms, ETL, governance. The university follows neither branch with any depth. The reason is structural. Introducing a new subject into a CS curriculum takes years of bureaucratic and administrative effort. Universities teach foundations because foundations are stable. Specific technologies move too fast for a system designed to change slowly. * SQL is foundational enough to make the cut * Snowflake, Databricks, dbt, Airflow are all “too specialized” * Anything resembling a real data engineering stack gets classified as professional training, outside the university’s scope Data Gibberish is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. The 10-Year Lag Is Built Into the System Mihail put it bluntly. Universities are roughly a decade behind the industry. By the time a discipline earns a permanent spot in the curriculum, the profession has already moved on to the next generation of tooling. The workaround at Plovdiv University is elective courses. Mihail and his colleagues created an entire data engineering track outside the required curriculum. Students who are dedicated sign up semester after semester and piece together the big picture on their own initiative. A full data engineering discipline exists in the master’s program, but only because individual lecturers pushed for it. The core curriculum still treats data engineering as a niche profession. The students who figure out it matters do so despite the system, not because of it. Hard Skills Used to Be Enough. They Aren’t Anymore. A few years ago, knowing one programming language and one framework got you hired. The COVID-era hiring boom rewarded narrow skill sets. Companies needed seats filled. They needed people who could write code in a specific technology and ship tickets. Thousands of junior developers entered the industry every month on the back of a single concentrated skill. No curiosity required and no product understanding expected. Know React, get a job. That worked because the economics supported it. The Market Shifted and the Craftsmen Got Stuck The economics changed. Companies stopped hiring for volume. AI tools absorbed the foundational knowledge work. People who built their entire career around one framework or one language found themselves competing with a chatbot that explains object-oriented programming better than most university courses. Information is everywhere now. What separates a valuable engineer from a replaceable one is everything around the information. * Understanding the product and the customer * Adapting when the tooling changes underneath you * Operating with judgment in ambiguous situations * Communicating with stakeholders who speak a different language than you do Universities Still Train for the Old Game The university teaches you how to use tools. The job requires you to understand why you’re using them and what problem they solve for the business. Mihail made a sharp distinction. Universities produce IT people, not developers. The degree gives you logic, math, and a surface-level tour of programming concepts. But the industry needs people who solve problems related to communication, technology, and economics. Those are three different skill sets, and a CS curriculum addresses one of them on a good day. The people who stall are the ones who treat graduation as the finish line. The ones who keep going treat it as a starting point for a much longer education that happens inside the business, inside the community, and inside the messy reality of real projects. Your Network Is Your Actual Career Engine The cliche says your network is your net worth. It holds true even when you look for a job. Blasting CVs across LinkedIn is the path of least resistance. It feels productive because you’re doing something. But it’s a low-leverage move that puts you in a pile with hundreds of other applicants who look identical on paper. Mihail’s company has never posted a single job offer on a popular platform. They hand-pick students directly from the university. Every hire comes through relationships built during lectures, projects, and community events. That pattern repeats across the industry more often than people realize. Show Up Where the Decisions Happen Conferences and user groups are where project leads, hiring managers, and senior engineers gather to talk about the work they care about. These people are approachable in that setting. They want to talk shop. The move is to ask about their problems, not their openings. * What kind of scaling challenges are you dealing with? * Where does delivery break down on your projects? * What does your data stack look like and where does it hurt? These questions signal curiosity and understanding. “Do you have any open positions?“ signals desperation. One leads to a conversation. The other leads to a polite nod and a business card that goes nowhere. Make Yourself Visible Before You Need a Job If you’re early in your career and don’t have deep experience to share, you still have options. A GitHub project, a blog post about a technology you’re learning, a short tutorial, a LinkedIn thread documenting what you figured out this week. None of these need to be brilliant. They need to exist. Visibility compounds. The people who run conferences and user groups see the same faces year after year. They remember your questions, and when a role opens up on their team, they think of the person they’ve been talking to for the last six months before they think of the 200 CVs sitting in their inbox. I found my current company through conferences. Several people I’ve worked with came from user groups and community events. This is how the industry works when you look past the job boards. Find Misho Here’s how you can find more about Mihail and his initiatives for data newcomers. Everything he does at this point is in Bulgarian, but Mihail promised to start building educational content in English, too. * LinkedIn profile * Snowflake user group * Free Snowflake course Closing Words The university gives you a starting line. Logic, math, a community of confused peers who share your ambition. That matters more than the cynics admit. But the finish line is somewhere else entirely. The skills that make data engineers valuable are built in the space between academia and industry. Pipelines, product thinking, stakeholder conversations, judgment under ambiguity. No curriculum covers this. No lecture hall prepares you for the moment a stakeholder asks you to scope something that has never been done before and expects an answer by Thursday. The people who figure this out early have an unfair advantage. They show up at conferences while still in their second year. They build side projects that prove curiosity, not mastery. They surround themselves with people who are ahead of them and ask questions until the vocabulary clicks. The people who wait for a curriculum to teach them what the job demands graduate with a degree and a gap where their career momentum should be. University is a foundation. Treat it as one. Build everything else yourself. — Yordan More on the Topic * The Surprising Trait That Beats Experience * Level-up Data Engineering * Why the Hard Skills Obsession Is Misleading Every Aspiring Data Engineer This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.datagibberish.com/subscribe

    59 min
  3. She Runs a $10k/Month Business Using Data and AI (and Her Title Says Marketing)

    APR 2

    She Runs a $10k/Month Business Using Data and AI (and Her Title Says Marketing)

    I invited Yana G.Y. to a live conversation because she does something rare. She has a director title and a Substack generating between $5K and $10K a month on the side of her full-time banking job. And she runs it entirely on data. Data-driven the way subscription businesses actually operate it. Numbers connected to decisions, decisions connected to outcomes. Below are the ideas she shared. The full conversation is in the video above. Stakeholders Make Emotional Decisions First (and They Have Good Reason To) The standard data professional complaint is that stakeholders ignore data and run on gut. Yana flips that framing. Her view: Most people are wired to make emotional decisions first and look for data that confirms them second. From her background in neurolinguistic programming, she puts the share of people who genuinely process the world through numbers at around 15%. Everyone else needs a different approach. Showing up with a dashboard and expecting it to land is wishful thinking. Data needs a story around it. A number without context gets interpreted differently by every person in the room. One sees growth, another sees underperformance, a third questions whether the metric tracks the right thing at all. Yana learned this early. Every conversation with engineering teams felt like friction. She was asking questions they thought she should already know. They were building things that missed what she actually needed. The gap cost both sides time. What closed it was her decision to learn enough about how systems work to ask better questions. To stop being helpless in technical conversations. The One Move That Ends Arguments in Any Meeting Yana has one technique she uses across every organization. It works in telecom. It works in banking. She has a perfect record with it. Before any contentious conversation, find the data that links your ask to what the organization actually cares about. Customers, revenue, profit. Those three. Then add competitive benchmarks if you can find them. In her current bank, she came in and asked how many successful app registrations they were getting. She pulled the internal rate, found competitor data, and discovered some competitors were achieving double or triple the registration conversion rate. She put that in front of the team. The conversation changed. People stop defending their position when the data shows the position is costing them. Go into the meeting with the numbers already pulled. Link them to something the people in the room care about. Then let the data do the work. This applies directly to data teams. When a project needs prioritization, funding, or unblocking, skip the technical justification. Show the business number it connects to, then show where a competitor is already ahead. Data Gibberish is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. What Yana Actually Tracks on Substack Substack’s native analytics fall short. Yana built her own spreadsheet and tracks the following every month: * Total subscribers * New subscribers * Total paid subscribers * New paid subscribers * Churned paid subscribers * New revenue * Average revenue per subscriber per month * Average revenue per subscriber per year * Free-to-paid conversion rate * Net Promoter Score The benchmark Yana uses for revenue per subscriber is the email marketing industry average, ranging roughly from $1 to $12 per subscriber per year depending on niche. Above average means the business is healthy. Below average means something needs auditing. The NPS tracking is the piece most Substackers skip. She uses Substack’s survey feature to ask paid subscribers one question: on a scale from 1 to 10, how likely are you to recommend this publication to someone you know. That one question, run through the standard NPS formula, tells her whether her paid tier delivers real value. Her target is above 70. Yana’s reasoning: NPS is the only reliable signal for whether paid subscribers are satisfied or are simply on their way out. How She Runs a $10K/Month Business in the Hours Left Over From a Full-Time Job Yana works five days a week, eight hours a day, in an office. She can answer Substack comments during lunch or late after work. That’s the window she operates in. AI handles everything operational. Formatting, research, scheduling, the mechanical parts. For writing, she trained a system on her own content. She extracted all her published articles into a database, connected it to an MCP server, and built what she calls a second brain. She can query it for content gaps, understand what paid subscribers want more of, and run analysis on her own business data. She writes in her voice, informed by her own data, with AI executing the work that requires no judgment. Her warning about AI: It pulls you into rabbit holes when you let it. Every answer suggests the next step, and the next step, and an hour later you have built something with no clear business purpose. Use it to execute a decision. Know exactly what you want before you open it. Yana also made the point that the 9-to-5 built the Substack. The telecom career gave her subscription business knowledge, customer journey expertise, conversion rate benchmarks, and an understanding of which metrics actually predict business health. All of it went directly into how she runs the newsletter. The side business grew because of the day job. Final Thoughts Yana has 15 years of subscription business experience, a disciplined tracking system, and a clear rule: link your ask to numbers that matter, then find the benchmark that shows where you stand. That’s the part worth sitting with. Data professionals spend time arguing that stakeholders ignore data. Yana learned enough about systems to have real conversations with engineers. She built her own analytics because the platform’s fell short. She tracks NPS because her industry background taught her it mattered. The gap between business and data closes when people on both sides decide to move. Yana moved. The engineers still telling business people to figure it out on their own are the ones falling behind. Interested into growing your Substack? Subscribe to Yana’s publication. This newsletter is funded by paid subscriptions from readers like yourself. If you aren’t already, consider becoming a paid subscriber to receive the full experience! See what people say about working with me. You are more than welcome to find whatever interests you here and try it out in your particular case. Let me know how it went! This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.datagibberish.com/subscribe

    56 min
  4. How to Think Holistically About the Data Ecosystem with Dylan Anderson

    MAR 19

    How to Think Holistically About the Data Ecosystem with Dylan Anderson

    Most data professionals are really good at one thing. They know their domain, they know their tools, and they go deep. That depth is how you get hired. It is also how you get stuck. The further you go in your career, the more you run into problems that don’t live inside a single domain. A pipeline that nobody uses because the data model underneath it was wrong. A dashboard that answers the wrong question because nobody talked to the business before building it. A data strategy that looks good on paper and falls apart when it hits the actual architecture. I sat down with Dylan Anderson , data strategist, consultant, and author of The Data Ecosystem newsletter, to talk about what it actually means to think across the system. Dylan came from business consulting before moving into data, which means he sees this from both sides. The conversation covered how companies actually operate, what separates the people who get ahead from the ones who stay stuck, and one habit that changes how you work without requiring you to become an expert in everything. Here is what I took away. The Data Ecosystem Is Not Your Tech Stack What most people call the data ecosystem is really a landscape, a snapshot of tools at a point in time. It changes occasionally, but it’s static. The ecosystem is something different. Think of a pond. Everything inside it is connected: the soil, the water, the microorganisms, the plants, the animals. Remove one element and the whole system shifts. Data works the same way. Your data model affects your engineering. Your engineering affects your analytics. Your governance affects your ML. Your business model shapes all of it. These aren’t separate domains you visit when a ticket arrives, but a system, and if you only know your corner of it, you’re operating blind. The problem is that most people work reactively. They handle what’s in front of them. They talk to the governance team when there’s a compliance issue, not because they thought about how governance connects to what they’re building. They don’t think about the relationships between domains. They just navigate them when forced to. Dylan’s point is that the people who think proactively about those connections, who anticipate how one decision ripples into another domain, are the ones who produce better work and get trusted with bigger problems. That’s a systems skill. And most data training never touches it. Paid members consistently share they got promoted or praised because they apply my guides. Most Companies Are Still a Mess Dylan has worked with a lot of billion-dollar organisations. His read: mostly chaos. Data strategy documents exist. They talk about vision and use cases. They don’t connect to the architecture underneath, the governance layer, or the engineering reality on the ground. Nobody in the building knows how all the pieces fit together, not even the CDO, who spends most of their time managing upward. The result is piecemeal data. Companies ahead in one domain are five steps behind in another. They buy great BI tools on top of poor data models and wonder why the dashboards don’t deliver. They bolt AI onto existing business processes and get some ROI but miss the structural shift entirely. What’s almost always missing is the link between business strategy and how data actually enables the organisation. And the people who see that link, who say “we need to think about this before we build that,” are the ones who get trusted, get promoted, and get asked into the room where decisions happen. Dylan’s observation from client work: the best performers in every organisation he’s walked into are the ones thinking across domains. LinkedIn will have you believe most companies are running cutting-edge data stacks. The reality for 95% of them is legacy chaos, and that gap is an opportunity for anyone willing to think holistically about it. Data People Need Business Literacy We obsess over data literacy. Get the business stakeholders to understand data, read the dashboards, ask better questions. Dylan flips it. Data people need business literacy just as much, and most of them don’t have it. They don’t know how the business model works, what the real KPIs are, why the marketing team cares about a number that looks meaningless from an engineering seat. They build things nobody uses because they never asked what outcome was actually needed. The gap is generational too. Five or ten years ago, most people who moved into data migrated from somewhere else, finance, operations, product. They understood business from the inside. Now whole cohorts start their careers as data people. They’ve never sat on the other side of the table. Dylan’s framing is direct: if you want to influence the business, you have to speak its language. That starts with understanding the business model, the KPIs, what each stakeholder is actually measured on. Once you have that, you stop presenting data and start presenting decisions. Stakeholders notice. They start saying you’re different from the engineers who only think about the technical problem. That’s when trust builds and trust is what gets you the work that matters. The “So What” Frame Dylan had a consulting partner early in his career who rejected every slide that didn’t answer one question: so what? Not “is this technically correct“, or “did you finish it“ So what. What changes because of this? What’s the action out of it? Dylan hated it at the time. It’s second nature to him now. It’s the fastest way to pull yourself out of ticket mode and into strategic thinking. You don’t need to study the whole data ecosystem to start thinking more holistically. You need one habit applied every time you deliver something: * Does this matter? * To whom? * What do they do next? If you can’t answer that, you’re producing output. The business is paying for impact. That question also protects you from the trap Dylan sees in most data teams, the reactive, firefighting, ticket-based mode that makes it almost impossible to step back. When every day is about closing tickets, you never ask whether the tickets matter. The “so what“ frame forces that pause. Fifteen minutes of strategic thinking before you build something is worth more than three days of building the wrong thing fast. What AI Changes and What It Doesn’t AI is taking over individual technical tasks faster than most people are comfortable admitting. Routine work, boilerplate code and first-draft outputs are largely handled. What’s left is context. AI doesn’t know your organisation. It doesn’t know why a specific stakeholder cares about a specific number, or whether a proposed solution fits your actual business model, or what you’ve already tried and why it failed. That’s your job. Domain expertise plus the ability to connect domains, to say “this decision over here affects that system over there“, is what makes AI output useful rather than generic. The people who will matter in the next five years are the ones who can think across the system. AI handles the execution. You handle the judgement. Dylan is also working on the next layer of the problem. How business models fundamentally change in an AI-first world. Most companies are bolting AI onto existing processes and getting some ROI. The shift is rethinking how the organisation operates at the root. It has implications for data modelling, governance, architecture, strategy. Nobody has solved that yet. It’s the open problem, and it’s where the real opportunity sits for anyone thinking holistically right now. Where to Find Dylan Dylan writes the Data Ecosystem newsletter on Substack. Weekly deep dives on every domain in the data space and how they connect. If you want to build the holistic view we talked about, start there. He also posts daily on LinkedIn. If you want to connect, include a note. He reads them and responds. And this summer he’s releasing a LinkedIn Learning course on insights-led thinking in the age of AI. Worth watching for. Final Thoughts The data industry has spent years rewarding depth. Know your tool, know your domain, go deep. That got a lot of people to senior. It stops working somewhere around there. The problems that matter at the senior level and beyond don’t live inside a single domain. They live in the connections between domains. Between the data model and the engineering. Between the engineering and the business outcome. Between what the team is building and what the organisation actually needs. The people who see those connections are the ones who get trusted with the hard problems. Business literacy, holistic thinking, asking “so what“ before you ship anything. None of this is complicated. It’s just not what most data training covers, so most people never build it deliberately. You can wait until your career stalls to figure that out. Or you can start now, while the gap between you and everyone else who only knows their lane is still wide enough to matter. Until next time, Yordan PS: Many paid members tell me they got promoted after working through the resources inside. If that sounds relevant to where you are right now, here’s what you get with a paid membership. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.datagibberish.com/subscribe

    48 min

  5. MAR 10

    How to Think like a Senior Engnieer with Erfan Hesami

    Seniority is a quiet death of the ego that thinks a perfect pull request solves the business problem. In this conversation, Erfan Hesami and I dismantle the myth that technical depth is the only path to the top. You spend years mastering Python and SQL only to reach a level where pipelines fail because requirements were wrong or stakeholders changed their minds. This is the ceiling where most data careers stall. Erfan, drawing from his experience in fintech and manufacturing, confirms that the jump to senior is not about a tenth tool. It is about shifting focus from the immediate task to the broader system. You must stop being a ticket-taker and start being an operator who designs for long-term reliability. Erfan highlights that the senior mindset requires moving beyond the IDE to master the non-code half of the job. This involves making hard calls on “good enough“ solutions and building trust through translation. We discuss how to bridge the gap between technical skill and leadership so your career does not stall. This session covers the specific mindset shifts and operational boundaries Erfan uses to cross the senior threshold. The Senior Operator Writes Less Code Erfan notes that mid-level engineers often measure value by the complexity of the systems they build. You want the newest framework because it feels like progress. Senior engineers realize every line of code is a liability. You provide more value by preventing a project than by building it poorly. When you operate at senior level, your primary job is making technical decisions aligned with business problems. You prioritize projects based on impact rather than technical interest. If a simple SQL view solves the problem, you do not build a custom microservice. The consequence of over-engineering is a maintenance debt that eventually prevents you from shipping anything new. Reframe your role from a builder to a decision-maker. You are there to ensure data flows reliably, not just to move bytes. Erfan suggests that a good week is not one where you closed the most tickets, but where you simplified an architecture or spent time upskilling to prevent future technical debt. Data Gibberish is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Ownership Is Moving Without Permission Waiting for a manager to tell you what to do is a junior behavior. Erfan observes that many competent engineers stay mid-level because they wait for a formal promotion before acting like a leader. They expect the title to grant them the right to speak up. In reality, the title follows the behavior. You must take ownership of the end-to-end lifecycle before anyone asks. Erfan emphasizes raising your hand when you see a flaw in database design or a better way to handle upstream sources. If you wait for permission, you are a resource. If you propose solutions and arrange meetings with stakeholders, you are a leader. The tradeoff is that ownership comes with accountability. When a project fails, you cannot blame the requirements. Erfan argues that unclear requirements are an engineering problem. You bridge this gap by networking across departments like CRM or accounting to understand pain points before you ever open your IDE. Zoom Out to See the System Juniors focus on the feature in front of them to move a ticket from left to right. This narrow focus leads to local optimizations that break the global system. Erfan warns against building a fast pipeline that produces garbage data because you did not check the upstream quality. As a senior, you must zoom out to see the broader system. Erfan explains that you must ask why a feature was requested and how it affects downstream consumers. You focus on the foundation that makes the platform scalable and easier for others to use over time. This shift in perspective allows you to catch errors in assumptions before they become code. This systemic thinking is your true differentiator. Erfan points out that AI can write boilerplate code, but it cannot navigate the politics of conflicting requirements. Your value lies in your ability to communicate and think through these nuances. The Resources Here are the links to all the resources we mentioned: * Pipeline To Insights: Erfan’s Substack newsletter where he shares his learning experiences and data engineering insights. * Erfan’s on LinkedIn: Here you can connect with Erfan and read his new * The GitHub Portfolio Article: A specific article mentioned that was written for Pipeline 2 Insights regarding how to build a GitHub portfolio. * Article on Certifications: A “very nice” article written by Jordan discussing the role and value of certifications in a data career. Final Thoughts The shift Erfan describes is a move from the certain world of syntax to the ambiguous world of people and systems. You must be willing to ask the questions that feel stupid but reveal deep flaws in a plan. You must be comfortable sharing opinions even when they are not explicitly requested. True seniority is not a reward for time served. It is a status earned by those who take responsibility for outcomes, not just outputs. Erfan’s approach shows that you stop being a person who writes code and start being a person who solves business problems using data. The code is just a tool, and often it is the most expensive one you have. Focus on building trust through consistency and ownership. Become the expert who can navigate different areas and adapt to new challenges. When you master the non-code half of the job, your career will stop stalling. Until next time, Yordan Data Gibberish is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.datagibberish.com/subscribe

    53 min
  6. Intro to MetadataOps with Alejandro Aboy

    MAR 5

    Intro to MetadataOps with Alejandro Aboy

    Alejandro Aboy and I sat down to talk about the mess of modern AI. He told me he feels like an octopus lately. He spent years mastering dbt and Airflow. Now his day involves managing agent protocols and context windows. The problem is that AI agents are the most demanding stakeholders you have ever had. They have zero intuition. They do not understand your tribal knowledge or your hidden business logic. If you give them a table without a deep layer of metadata, they will guess. When they guess, they hallucinate. When they hallucinate on your watch, you lose your seat at the table. This conversation with Ale highlighted that the job has changed. You are no longer just shipping rows, but shipping the instructions that keep the machines from lying. The Death Of The Static Catalog Traditional data catalogs are where documentation goes to die. You spend a month tagging columns and then nobody looks at them. Ale calls the new approach MetadataOps. In this model, your agents provide data back to the system. They help you understand how they use the information. If an agent struggles to identify a primary key, that is a production bug. Documentation is now the fuel for your production AI. If you treat metadata as an afterthought, your AI solutions will stay in the playground. Real operators build systems that document themselves through use. Data Gibberish is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Managing The Unstructured Mess Unstructured, human generated data like images, audio or markdown documentation is a real mess. You must build guardrails that expect the mess. This means using regular expressions and specific skills to pull structure out of chaos. You are acting as a bridge between human habits and machine requirements. This requires deciding when a data source is “good enough” to ship. You have to weigh the risk of a hallucination against the speed of the business. These are leadership calls, not technical ones. The AI Stakeholder Shift Your stakeholders think everything is possible now because they saw a demo on Twitter. This creates a massive gap in expectations. People expect you to ship features ten times faster. They do not see the infrastructure required to make those features reliable. You must translate the technical risk of AI into business terms. If you fail to manage the context, you fail to manage the trust. You are a translator now. You explain why a simple prompt is not a production strategy. You show them that the quality of the output depends on the quality of the metadata you have been screaming about for years. Links We mentioned a few resources. Here are the links: * Alejandro’s publication: Make sure you subscribe. * Substack Recapped: A must try if you write on Substack. Final Thoughts The gap between a senior individual contributor and a data leader is the ability to manage ambiguity. Ale’s shift from dbt to agent protocols is a map for your own career. The code is becoming a commodity. The ability to structure the world for a machine is the new high-value skill. You cannot hide in the IDE and hope the data stays clean. You have to lean into the mess of unstructured files and vague stakeholder requests. Metadata is the only tool that bridges that gap. You need to start thinking about it as managing the context of your entire company. If you do not own the metadata, you do not own the outcome. Seniority is about taking ownership of the results, not just the scripts. Use your metadata to build a system that thinks. Until next time, Yordan from Data Gibberish PS: We live stream these conversations. Join and chat with us next time. Data Gibberish is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.datagibberish.com/subscribe

    1h 2m
  7. This Engineer Teaches Their Audience How To Think The Right Way

    JAN 20

    This Engineer Teaches Their Audience How To Think The Right Way

    The session centers on a fundamental problem: You and I are trained as engineers to use design patterns for code, but we’re often left winging it when it comes to people and organizational complexity. Michał Poczwardowski argues that we need to treat our thinking process with the same level of structural integrity we give our architecture. Mental Models are Just Design Patterns for Your Brain Michal defines mental models as “design patterns for solving problems“ rather than just writing code. They are thinking protocols pulled from disciplines like physics and chemistry that help you ask the right questions before you commit to a path. The goal is to have a meta-observation of your own brain so you don’t default to lazy or reactive thinking. The Logic of “First Principles” and “Inversion” * First Principles: Instead of following “best practices“ blindly, you break a problem down to its fundamental truths. If the math doesn’t work at the foundation, the whole project is a house of cards. * Inversion: Instead of asking “How do I make this project succeed?“, you ask “What are all the ways we could make our people miserable and the project fail?“. Then, you systematically avoid those things. It’s much easier to avoid stupidity than to seek brilliance. The “Process vs. Outcome” Fallacy This is where most engineers get tripped up. Michal pointed out that a good decision can lead to a bad outcome because of luck or unexpected variables. Conversely, you can make a “stupid“ decision, like betting the company’s runway on a random crypto coin, and get lucky. You cannot judge the quality of your decision-making based solely on the result. You have to judge it based on the structure of the process you used at the time. If the process was solid but the result was bad, you didn’t fail; you just hit an outlier. Practical Protocol: Don’t Estimate on the Fly One of the best “thinking protocols“ mentioned was the rule of never providing estimates during a live call. The pressure to look smart or accommodate a client usually leads to “one-week“ promises for “two-month“ problems. A structured thinker pauses, analyzes, and responds in a separate thread. Data Gibberish is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Thanks for reading, Yordan PS: Want to learn more about structured decision making and mental models? Subscribe to Perspectiveship today. Let’s connect LinkedIn | Threads | Facebook | X | Bluesky This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.datagibberish.com/subscribe

    43 min
  • 7 Episodes

About

Data Gibberish helps experienced data engineers and data leads handle stakeholder requests, scope pressure, and decision-making with scripts, checklists, and playbooks you can use immediately. www.datagibberish.com

Information

  • Creator
    Yordan Ivanov
  • Years Active
    2K
  • Episodes
    7
  • Rating
    Clean
  • Copyright
    © Yordan Ivanov
  • Show Website
    Data Gibberish