TestGuild Automation Podcast

Joe Colantonio

TestGuild Automation Podcast (formally Test Talks) is a weekly podcast hosted by Joe Colantonio, which geeks out on all things software test automation. TestGuild Automation covers news found in the testing space, reviews books about automation, and speaks with some of the thought leaders in the test automation field. We'll aim to interview some of today's most successful and inspiring software engineers, and test automation thought leaders.

  1. 11h ago

    The AI Illusion: Why Testing Still Needs Humans with Joe Colantonio

    Everyone is talking about AI replacing testers, writing tests, and transforming software quality. But what if we're asking the wrong question? In this solo episode, Joe Colantonio shares a growing concern he's seen while traveling across the country for TestGuild IRL events: a decline in testing fundamentals at the exact moment AI hype is reaching a fever pitch. Drawing insights from Carissa Véliz's book Prophecy: Prediction, Power, and the Fight for the Future, Wayne Roseberry's work on AI and meaning, and Tariq King's concept of Human Experience Testing, Joe explores why AI systems may be far less intelligent than many believe, and why human testers remain more important than ever. You'll discover: ✅ Why large language models generate plausible answers without understanding truth ✅ The difference between prediction, correlation, and genuine understanding ✅ Why AI can test software but cannot experience software ✅ What "Everything is tested, but nothing is experienced" really means ✅ How AI hype may be distracting teams from critical testing fundamentals ✅ Why empathy, context, and human judgment are becoming competitive advantages for testers Whether you're excited about AI, skeptical of it, or somewhere in between, this episode will challenge you to think more deeply about the future of testing and your role in it. Resources Mentioned 📖 Prophecy: Prediction, Power, and the Fight for the Future by Carissa Véliz 📖 Work and presentations by Wayne Roseberry 🎓 Free course: thebullshitmachines.com 🎤 Learn more about TestGuild IRL events: TestGuild.com/irl If you enjoy this episode, be sure to subscribe, leave a review, and share it with a fellow tester who's trying to navigate the AI era without losing sight of the fundamentals. #SoftwareTesting #AI #QualityEngineering #TestAutomation #SoftwareQuality #HumanExperienceTesting #ArtificialIntelligence #TestGuild #QA #TechPodcast

    16 min

  2. Jun 2

    AI Agents in QA: How to Keep Up with AI-Driven Dev Velocity with Vilhelm von Ehrenheim

    AI coding tools promised to make development faster — and they delivered. But here's the problem nobody talks about enough: when you speed up coding, you don't eliminate the bottleneck in the SDLC. You just move it. And for most teams, it lands squarely in QA. In this episode, Joe sits down with Vilhelm von Ehrenheim, Co-founder and Chief AI Officer of QA.tech, to dig into how agentic AI is reshaping software testing from the ground up. Vilhelm brings serious ML credibility, he helped build Motherbrain, one of the earliest production LLM systems in venture capital, and he's now applying that experience to one of the hardest problems in software delivery: testing at AI development velocity. You'll learn how QA.tech's behavioral knowledge graph gives AI agents the context they need to actually understand your application, why validating user intent beats checking element identifiers every time, how autonomous agents can review PRs, reproduce bugs from Slack messages, and generate targeted tests without a single line of test code ,and what the tester's role actually looks like when agents do the heavy lifting. If you're wondering whether your QA practice can survive the pace of AI-driven development, this one's required listening. 🔗 Book a demo now: https://testgld.link/qatechdemo

    35 min

  3. May 19

    Maestro MCP, AI Mobile Testing That Fixes Its Own Tests with Leland Takamine

    What happens when AI agents can not only write mobile app code, but also validate their own work automatically? In this episode, I sit down with Maestro Co-founder and CEO Leland Takamine to explore one of the biggest shifts happening in software testing right now: agentic mobile testing. Leland shares how his team went from solving mobile performance testing challenges to building one of the fastest-growing mobile automation frameworks used by companies like Microsoft, Meta, Amazon, and DoorDash. We dive deep into: How AI coding agents are changing mobile testing workflows What "closing the agentic feedback loop" actually means Why deterministic testing still matters in the age of AI How Maestro MCP lets AI agents validate mobile app changes automatically Why mobile test maintenance may finally become manageable The future role of testers as AI-generated code explodes Leland also gives a live demo showing an AI agent building, validating, debugging, and generating a reusable mobile test completely autonomously. If you care about AI testing, mobile automation, MCP servers, or the future of QA engineering, this episode will likely change how you think about testing workflows over the next few years. Try it out now for yourself:  Maestro Studio: https://testgld.link/mstudio Maestro MCP docs: https://testgld.link/maestromcp

    38 min

  4. May 12

    AI Testing Costs, How to Prevent Runaway Token Bills with Arthur Hicken

    AI-powered testing tools are exploding across software engineering teams… but so are the hidden costs. In this episode, Joe sits down with Arthur Hicken to unpack the growing problem of runaway AI token usage, unexpected LLM billing, and the operational risks of deploying AI agents into testing and DevOps pipelines. Inspired by Arthur's article on the emerging "Token Tax," this conversation explores why many teams are underestimating the true cost of AI automation. You'll learn: Why AI-generated testing can create unexpected scaling costs How runaway AI agents and infinite loops happen Real-world examples of massive AI billing surprises Why deterministic problems shouldn't always use LLMs The hidden risks of "vibe testing" and autonomous AI remediation How QA teams can monitor, test, and control token usage Why performance testing and service virtualization matter more than ever in AI systems Practical strategies to avoid expensive AI deployment mistakes Whether you're a software tester, automation engineer, QA leader, or DevOps practitioner, this episode will help you think more strategically about AI testing before costs spiral out of control.

    39 min

  5. May 5

    Stop Rewriting Tests: How to Add AI to Selenium and Playwright Without Starting Over with Alex Rodionov

    AI-powered testing tools promise faster automation and less maintenance, but most require teams to abandon their existing frameworks. In this episode, we explore Alumnium, an opensource AI-native end-to-end testing solution created by Alex Rodionov, an engineer at Airbnb and a tech lead on the Selenium project. Instead of replacing tools like Playwright or Selenium, Alumnium adds an AI layer on top, helping teams: Reduce test maintenance by removing brittle locators Build more resilient, self-healing tests Write less code while increasing coverage Run tests across web and mobile using intent-based steps We also go beyond the hype and break down what actually matters for real teams: Why AI-driven tests can still become flaky The performance and cost tradeoffs of LLM-based execution What "context rot" is—and how it impacts long test runs How to safely introduce AI into existing test suites without rewriting everything Check it out now: https://testguild.me/alumAI

    30 min

  6. Apr 28

    Claude AI Mobile Testing, Run Real Device Tests with AI with Frank Moyer and Chris Faulhaber

    AI is changing how we build and test software, but most teams are still struggling to turn AI-generated tests into real production value. Use code TESTGUILD3 try for yourself free now for 3 months: https://links.testguild.com/Kobiton In this episode, we break down what actually works when it comes to AI-powered mobile test automation, especially when running tests on real devices not simulators from Claude. You'll learn: How teams are generating and running Appium tests using natural language in minutes Why AI-generated tests often fail—and how to avoid costly false positives The real impact of AI on test automation roles and responsibilities How real device testing exposes issues AI alone can't catch Practical ways to reduce test maintenance while increasing coverage We also explore a major shift happening right now: AI is making it easier to create tests—but dramatically increasing the volume of code and risk that needs to be validated. That means one thing: Testing isn't going away—it's becoming more critical than ever. If you're a QA engineer, automation engineer, or DevOps leader trying to keep up with AI-driven development, this episode will give you a clear, practical perspective on what to focus on next.

    31 min

  7. Apr 15

    AI Testing Is Breaking Your Pipeline. Fix Quality Before It's Too Late with Eric Minick

    AI coding tools are helping teams move faster than ever, but there's a hidden cost. In this episode, we break down new insights from a DevOps industry report revealing a growing "velocity paradox": teams are shipping more code, but experiencing more failures, rollbacks, and burnout. You'll discover why AI adoption is heavily skewed toward coding, but not testing, pipelines, or observability, and how that imbalance is creating fragile systems that break under pressure. More importantly, you'll learn what high-performing teams are doing differently to maintain quality while scaling speed. What You'll Discover: ✔️ Why AI is increasing deployment failures (and how to stop it) ✔️ The "velocity vs quality" trap hurting modern DevOps teams ✔️ How to reduce flaky tests and pipeline instability ✔️ Why observability and feature flags are now critical, not optional ✔️ Practical ways to improve your CI/CD pipeline for AI-driven development ✔️ The role of QA engineers in the age of AI (and why it's growing, not shrinking) If you're a tester, automation engineer, or DevOps leader trying to keep up

    30 min

  8. Apr 7

    Scaling Quality Engineering: How to Deliver Faster Across Global Teams with Sunita McCoy

    AI is changing how we build and test software, but most teams are struggling to turn that promise into real results. In this episode, we break down what it actually takes to scale quality engineering across global teams without creating bottlenecks, burnout, or broken processes. You'll learn: why most test automation and transformation initiatives fail how to separate AI hype from reality what high-performing teams are doing differently to ship faster with confidence Today's expert, Sunita McCoy, a Global Engineering Leader and Transformation Specialist, shares practical insights from leading large-scale engineering transformations, including: how to build a culture that supports AI adoption why "quality as a phase" is dead how to shift toward treating quality as a product If you're a QA leader, automation engineer, or DevOps professional trying to improve reliability, reduce risk, and future-proof your skills in the age of AI, this episode gives you a clear path forward.

    34 min
See All (596)

4.9
out of 5
102 Ratings

About

TestGuild Automation Podcast (formally Test Talks) is a weekly podcast hosted by Joe Colantonio, which geeks out on all things software test automation. TestGuild Automation covers news found in the testing space, reviews books about automation, and speaks with some of the thought leaders in the test automation field. We'll aim to interview some of today's most successful and inspiring software engineers, and test automation thought leaders.

Information

You Might Also Like