Outperform

Datadog

Outperform is a show dedicated to unlocking the full potential of your business and product by scaling experimentation for data-driven decisions and innovation. Every episode is packed with cutting-edge strategies and practical insights for data scientists, PMs, and engineers alike. We’re exploring topics ranging from A/B testing and AI to creating a culture of experimentation within your organization. Discover how businesses win by driving innovation and better decision-making!

الحلقات

  1. قبل يومين

    Why Soft Skills Make or Break Experimentation Programs

    Most experimentation programs don't stall because the stats are wrong. They stall in the room when "the data says so" isn't enough to get a marketer to fix tracking, free up engineering time, or talk a CEO out of shipping a redesign with no test plan. This episode is about the part of the job no data science course covers: the soft skills that decide whether your program ever gets off the ground. Shiva Manjunath, host of the experimentation podcast ‘From A to B’ , joins host Ryan Lucht to reframe those skills as capital, the trust and goodwill you build with colleagues, then spend every time you push for a test or challenge a HiPPO. But they don't let soft skills off the hook either, because the same skills that win a room can just as easily sell a bad conclusion, which is why hard skills have to keep them honest. What follows is a candid, practitioner-to-practitioner take on why being right and being effective aren't the same thing. In this episode, we discuss: Why a "culture of experimentation" is really a culture that supports experimentation, and how early wins and internal champions build it How to spend your career and relationship capital wisely, from picking your battles to knowing when to disagree and commit Why data teams so often get out-maneuvered by better communicators, and how to close that gap without becoming one How to turn a redesign with no test plan into a "we'll test it once we're live" win

  2. ٤ مارس

    How Do You A/B Test SEO and GEO? Title Tags, Traffic, and the Truth

    What does it really mean to run SEO as an experiment instead of a checklist? In this episode, we’re joined by Will Critchlow and Mat Reiss to explore the evolution of SEO and AEO/GEO A/B testing — from enterprise-level experimentation frameworks to a super simple way for any team to get started: title tags. Will breaks down the fundamentals of SEO and AEO/GEO testing and how it differs from traditional user-based A/B testing. Because Google effectively acts as the “audience,” experiments must split pages rather than users and be implemented server-side so crawlers can see the changes. We discuss why organic traffic is a stronger measure of impact than rankings alone, how counterfactual forecasting helps isolate true lift, and why modern SEO increasingly feels like experimenting against a black-box, AI-driven system. Then I called up Mat to help us get real practical, really quickly — with “title tags”, what he calls the “last mile” of search performance. He explains why small copy changes can meaningfully increase click-through rate without improving rankings, shares a case study that drove a 30% traffic lift from a simple tweak, and outlines how teams of different sizes can begin testing title tags without heavy engineering resources. In this episode: How SEO and AEO/GEO A/B testing differs from traditional experimentation Why organic traffic is often a more reliable KPI than rankings The statistical and structural realities of page-based testing Why title tags are an underused but high-impact growth lever A real-world example of a 30% traffic lift from a simple title change When SEO testing becomes viable for small and mid-sized e-commerce teams

حول

Outperform is a show dedicated to unlocking the full potential of your business and product by scaling experimentation for data-driven decisions and innovation. Every episode is packed with cutting-edge strategies and practical insights for data scientists, PMs, and engineers alike. We’re exploring topics ranging from A/B testing and AI to creating a culture of experimentation within your organization. Discover how businesses win by driving innovation and better decision-making!