Inference & Intelligence Lab

Lin Jia

Inference & Intelligence Lab is a podcast on statistical inference, causal inference, machine learning, and GenAI evaluation, focused on making decisions that hold up in real-world data science. The show features two series—Causal Inference From the Ground Up and Inference in the Wild—covering both first principles and practical pitfalls.

Episodes

  1. Causal Inference from the Ground up EP3: Ladder of Causation: How to Upgrade from Prediction to Policy

    6 DAYS AGO

    Causal Inference from the Ground up EP3: Ladder of Causation: How to Upgrade from Prediction to Policy

    Headline: Why your "Perfect" Models are failing the Boardroom. For months, the team worked on the model. The AUC was 0.92. The validation sets were clean. But when you shipped it to the real world? Nothing happened. The metrics didn't move. The business didn't grow. Well, you didn't have a "data" problem. You had a Causality problem. You just hit the Data Validity Cliff—the point where your high-tech models are great at "seeing," but completely blind at "doing." In this episode, we break down Judea Pearl’s Ladder of Causation to fix your strategy: The Selection vs. Policy Trap: Why finding users who love you isn't the same as making users love you. The Owl vs. The Scientist: Why "superpowered observation" is a dead end for decision-making. The UK-to-US Simulation: How to "transport" data across markets without a new pilot. Stop being just an observer. Start becoming a simulator. 📖 Read the companion deep dive (with diagrams & checklists) : ⁠⁠https://inferenceintel.substack.com/p/ladder-of-causation-how-to-upgrade About the Host Lin Jia is a Senior Data Scientist and Craft Lead at Booking.com with over 9 years of experience. Operating at the intersection of statistical inference, causal machine learning, and GenAI evaluation, she specializes in building the frameworks that enable trustworthy, decision-ready insights under real-world constraints. A recognized expert in the field, Lin has authored research on sensitivity analysis presented at KDD 2024 and leads the development of organization-wide standards for experimentation and observational causal inference. Connect with me: Lin on ⁠⁠⁠LinkedIn: ⁠https://www.linkedin.com/in/linjia/⁠Join Inference & Intelligence Lab biweekly ⁠https://inferenceintel.substack.com/⁠

    16 min
  2. Causal Inference from the Ground up EP2: The Bridge to Truth: Why Identification Comes Before Estimation

    8 FEB

    Causal Inference from the Ground up EP2: The Bridge to Truth: Why Identification Comes Before Estimation

    The Infinite Data Trap: Why More Data Won't Save Your Causal Models You have petabytes of user data. Your model has 99% validation accuracy. But when you ask it, "What happens if we change our strategy?", it gives you an answer that is confidently wrong. Welcome to the Infinite Data Trap. In this episode, we reveal why "big data" is useless for decision-making without a critical, often neglected step: Identification. We break down the "Bridge to Causal Truth" and explain why identification is the only thing standing between a reliable insight and misinformation masquerading as truth. In this episode, we discuss: Identification vs. Estimation: Why moving from a "what-if" question to a concrete number requires a bridge of assumptions—not just a better algorithm. The E-commerce Blindspot: A real-world look at how unobserved user engagement can make a promotional email look like a success when it’s actually a bias. Beyond Correlation: How to be explicit about your assumptions (like "no interference") and why sensitivity analysis is your best defense against being "confidently wrong." Stop gathering data. Start identifying effects. 📖 Read the companion deep dive : ⁠ https://inferenceintel.substack.com/p/the-bridge-to-truth-why-identification About the Host Lin Jia is a Senior Data Scientist and Craft Lead at Booking.com with over 9 years of experience. Operating at the intersection of statistical inference, causal machine learning, and GenAI evaluation, she specializes in building the frameworks that enable trustworthy, decision-ready insights under real-world constraints. A recognized expert in the field, Lin has authored research on sensitivity analysis presented at KDD 2024 and leads the development of organization-wide standards for experimentation and causal inference. Connect with me: Lin on ⁠⁠⁠LinkedIn: ⁠⁠https://www.linkedin.com/in/linjia/⁠⁠Join Inference & Intelligence Lab biweekly ⁠⁠https://inferenceintel.substack.com/⁠

    12 min
  3. Inference in the Wild EP1: The DoubleML Ranking Disaster: Why PLR Fails for Multiple Discrete Treatment

    1 FEB

    Inference in the Wild EP1: The DoubleML Ranking Disaster: Why PLR Fails for Multiple Discrete Treatment

    The Ranking Trap: Why PLR Fails with Multiple Treatments You’re testing four different promotional strategies—a discount, free shipping, BOGO, and loyalty points. You run a Partially Linear Regression (PLR), get a clean table of coefficients, and rank them to decide your next big investment. There’s just one problem: Your ranking is likely a lie. In this episode, we dive into a dangerous "trap" in observational causal inference. We explore why regression-style estimators like PLR don't actually estimate the Average Treatment Effect (ATE) when effects are heterogeneous. Instead, they estimate a variance-weighted average that can completely reverse your treatment rankings. In this episode, we discuss: The Overlap Weighting Problem: Why PLR only "learns" where it has leverage, and how that creates a different subpopulation for every treatment you test. PLR vs. IRM: Why the Interactive Regression Model (IRM) is the essential choice for multiple discrete treatments. The Bridge to DoubleML: How to use machine learning to model "nuisance" components while maintaining statistical rigor. RCT vs. Observational Realities: Why PLR works in a randomized trial but fails in the "wild" of marketplace data. Stop letting overlap redefine your estimand. Start using the right model for the job. 📖 Read the companion deep dive : ⁠ ⁠ https://inferenceintel.substack.com/p/launching-the-inference-and-intelligence About the Host Lin Jia is a Senior Data Scientist and Craft Lead at Booking.com with over 9 years of experience. Operating at the intersection of statistical inference, causal machine learning, and GenAI evaluation, she specializes in building the frameworks that enable trustworthy, decision-ready insights under real-world constraints. A recognized expert in the field, Lin has authored research on sensitivity analysis presented at KDD 2024 and leads the development of organization-wide standards for experimentation and observational causal inference. Connect with me: Lin on ⁠⁠⁠LinkedIn: ⁠⁠https://www.linkedin.com/in/linjia/⁠⁠Join Inference & Intelligence Lab biweekly ⁠⁠https://inferenceintel.substack.com/⁠

    18 min
  4. Causal Inference from the Ground up EP1: The Core Trio of Causal Inference & The Art of Baking a Causal Cake

    12 JAN

    Causal Inference from the Ground up EP1: The Core Trio of Causal Inference & The Art of Baking a Causal Cake

    The Recipe for Causal Truth: Estimand, Estimator, and Estimate You have a dataset, a model, and a final number. But can you explain—with precision—what that number actually represents? In the world of causal inference, precision is everything. We often use terms like "the result" or "the algorithm," but failing to distinguish between the What, the How, and the Result is where most analytical errors begin. In this episode, we break down the three fundamental pillars of the estimation process: the Estimand, the Estimator, and the Estimate. Using a simple baking analogy, we demystify these academic terms and turn them into a practical framework for your daily data work. In this episode, we discuss: The Estimand (The "What"): Why defining your theoretical "ideal cake" is the most important step before touching any data. The Estimator (The "How"): How your recipe—the algorithm or function—transforms raw inputs into insights. The Estimate (The "Result"): Understanding the concrete number that comes out of the oven, and why it’s only an approximation of the truth. The Estimation Process: How to align these three trios to ensure your business questions are actually being answered by your code. Don't just run models. Understand the ingredients of your causal claims. 📖 Read the companion blogpost : ⁠⁠⁠ ⁠⁠ https://inferenceintel.substack.com/p/causal-inference-from-the-ground About the Host Lin Jia is a Senior Data Scientist and Craft Lead at Booking.com with over 9 years of experience. Operating at the intersection of statistical inference, causal machine learning, and GenAI evaluation, she specializes in building the frameworks that enable trustworthy, decision-ready insights under real-world constraints. A recognized expert in the field, Lin has authored research on sensitivity analysis presented at KDD 2024 and leads the development of organization-wide standards for experimentation and observational causal inference. Connect with me: Lin on ⁠⁠⁠LinkedIn: ⁠⁠⁠https://www.linkedin.com/in/linjia/⁠⁠⁠Join Inference & Intelligence Lab biweekly ⁠⁠⁠https://inferenceintel.substack.com/⁠

    13 min
  • 4 Episodes

About

Inference & Intelligence Lab is a podcast on statistical inference, causal inference, machine learning, and GenAI evaluation, focused on making decisions that hold up in real-world data science. The show features two series—Causal Inference From the Ground Up and Inference in the Wild—covering both first principles and practical pitfalls.

Information