AI-First Engineering Podcast

TestGrid

The podcast where engineering and technology leaders share how AI is transforming software delivery, quality, and the future of engineering

Episodes

  1. Oct 16

    Test Automation Is Solved. So Now What?

    Episode 8: Test Automation Is Solved. So Now What?Guest: Richard Bradshaw Host: Jon Robinson AI can generate tests, review code, read requirements, and explore applications faster than ever. But does more automation actually mean better quality? In Episode 8 of AI First Engineering, Jon Robinson talks to Richard Bradshaw, a long-time testing and quality engineering practitioner, speaker, and trainer, about why the industry needs to rethink its approach to test automation in the age of AI. Richard argues that traditional test automation is a solved problem. The bigger challenge is knowing what to automate, why it matters, and whether it delivers meaningful value. Adding AI to existing practices without questioning them could simply mean automating the wrong things faster. The conversation explores how AI is changing testing, the role of human judgment, and the opportunity to use AI beyond the test suite itself to gather information and help teams make better decisions. In this episode, we discuss: Why test automation is a solved problem: Why teams need a testing strategy first, with automation serving that strategy.Automating the wrong things faster: How chasing test counts and coverage can increase maintenance, execution time, flaky tests, and false positives.Going back to first principles: Why we test, what information we need, and how that information helps stakeholders make decisions.What makes a good test: Why context matters and why good practices depend on the team, product, and situation.AI beyond the test suite: How AI can process documentation, logs, metrics, and customer feedback to help teams find insights faster.The human role in AI-driven testing: Why human judgment, context, and responsibility remain essential, even when AI generates tests or explores applications.The future of Quality Engineering: How testing, development, and quality responsibilities may increasingly overlap, and why governance and good engineering practices matter.Richard also shares why the real opportunity is not simply to create more tests, but to use automation and AI to free people up, gather useful information faster, and make better decisions about quality. About the guest Richard Bradshaw has spent nearly two decades in testing and software engineering, contributing to the testing community through speaking, training, and community building. He has also been developing production-level AI agents and is preparing to launch Evolving Quality. Listen to the episode and join the conversation about what test automation should look like in an AI-first engineering world.

  2. Oct 7

    Is AI Rewriting the Rules of Agile Testing?

    Is AI Rewriting the Rules of Agile Testing?AI is changing software delivery fast, but this conversation argues the biggest factor is still the same: strong engineering habits, clear collaboration, and human judgment. Jon Robinson talks with testing veteran Lisa Crispin about what AI really changes in agile testing, where it helps, and where it can quietly make weak teams worse. They cover the shift from “agile testing” to Lisa’s broader “holistic testing” framing, why small batches still matter, and why quality is a whole-team responsibility even when AI generates code and assists with validation. Key Topics Lisa Crispin explains why AI feels different from past shifts like automation, DevOps, and continuous delivery: the pace is much faster, creating cognitive overload and “comprehension debt.”AI does not replace good engineering habits. It amplifies whatever already exists in a team.Strong teams using small batches, fast feedback, cross-functional collaboration, and shared quality ownership can use AI to move faster and do more thought work.Weak teams that already work in huge batches and rely on siloed handoffs may find AI makes the mess bigger by generating more code than they can understand or validate.Lisa explains why she and Janet Gregory moved away from the word “agile” and toward “holistic testing” because the term agile has become too overloaded and inconsistent.The conversation revisits the original intent of the Agile Manifesto and how “working software over comprehensive documentation” was often misread as “no documentation.”Lisa argues that documentation still matters, especially when teams need searchable context, continuity, and shared understanding.They discuss how AI can support collaboration and exploratory thinking, but can also create a false sense of collaboration when people start treating an LLM like a teammate.The episode emphasizes that psychological safety is essential for quality work, especially when testers need to raise concerns about builds, risk, or product direction.Lisa makes the case that testers are still needed, but the modern tester’s role is increasingly about coaching, enabling, and helping others learn testing and quality skills.Connect with Lisa Lisa’s website: LisaCrisman.comLisa on LinkedIn: Search for Lisa Crispin on LinkedIn.Agile Testing Fellowship:  Lisa mentions its newsletter, blog posts, and video chats with Janet Gregory.Follow AI First Engineering for more interesting discussions  https://www.linkedin.com/showcase/ai-first-engineering/ www.TestGrid.io

  3. Sep 16

    Did Playwright Win the AI Testing War?

    Episode 6: Did Playwright Win the AI Testing War?Playwright has quickly become one of the most talked-about tools in test automation. But has it actually won the AI testing war? In Episode 6 of AI First Engineering, Jon Robinson sits down with Andrew Knight, the Automation Panda, to explore why Playwright has gained so much momentum, how AI is changing test automation, and what this means for teams still working with Selenium. The conversation goes beyond the Playwright vs. Selenium debate and looks at test maintenance, developer experience, AI-assisted automation, mobile testing, and the increasingly important build vs. buy question. In this episode, we discuss:• Why Playwright is gaining momentum and how its developer experience and AI integrations are influencing adoption. • Whether Selenium teams actually need to migrate to Playwright, or whether existing reliable and maintainable frameworks can continue to serve their needs. • How AI is changing the way tests are created and maintained, and why standardized frameworks can make it easier for AI to work effectively. • Where Playwright fits in web and API testing, and where teams still need other tools for native mobile testing. • Why test maintenance remains a major challenge with highly customized Selenium frameworks. • Build vs. buy: When does building your own automation infrastructure stop making sense? Andrew discusses friction points including orchestration, execution resources, reporting, dashboards, and devices. • How AI could change the role of testing, with testing potentially becoming more of a shared responsibility across engineering and product teams. • Whether acceptance and evaluation could become increasingly important as teams rethink how quality is owned. About Andrew KnightAndrew Knight, known as the Automation Panda, has spent his career in software testing, quality, and automation. He started as an SDET and progressed into team leadership across multiple companies and industries. He has contributed to open source, including Boa Constrictor, a .NET implementation of the Screenplay pattern. Andrew has also worked in developer relations at Applitools and served as Director of Test Automation University. Today, he is Senior Director of Product and Engineering at Cycle Labs, building test automation solutions for supply chain and warehouse management systems. He is also a Playwright Ambassador, speaker, and educator focused on test automation, Playwright, AI, and software quality. Find Andrew🌐 Automation Panda: automationpanda.com 🌐 Auto Panda Motive: autopandamotive.com 💼 LinkedIn: Andrew Knight / Automation Panda 📚 LinkedIn Learning: Playwright and AI courses Key Takeaways• Playwright's developer experience and AI integration have helped drive its growing momentum. • Teams with reliable Selenium automation do not necessarily need to rewrite it. • AI can accelerate test creation and maintenance, but the underlying framework and consistency still matter. • Playwright is focused primarily on web testing and does not replace native mobile testing. • Build vs. buy depends on a team's skills, infrastructure, scale, orchestration, reporting, and execution needs. • AI may reduce manual testing work while making quality expertise and judgment more important. Listen to Episode 6 of AI First Engineering: Did Playwright Win the AI Testing War?

  4. Sep 2

    Will AI Agents Redefine the Software Development Lifecycle?

    Episode 5: Will AI Agents Redefine the Software Development Lifecycle?What happens when AI stops being a coding assistant and starts becoming part of the engineering team? In this episode of AI-First Engineering, Jon Robinson, Head of Market Development at TestGrid, talks with Piyush Tiwari, Senior Software Engineering Manager at Wayfair, about how AI agents are changing the way software gets built and what that means for the engineers building it. Piyush shares how his team has moved beyond traditional AI copilots. Instead of engineers simply using AI to write code, agents can now pick up a user story, work through it end-to-end, and bring an engineer in for review. The engineer's role starts to shift from being the person executing every task to becoming the director working with the agents. But that shift creates a bigger question: if AI takes over the grunt work, how do the next generation of engineers develop the experience needed to become senior engineers? Jon and Piyush explore: From copilots to agents: How AI has evolved from helping engineers write better code to executing work across the development lifecycle.The changing role of engineers: Why system understanding, judgment, and decision-making could become more important than writing code.The junior engineer problem: If engineers no longer learn through repetitive work, failures, and production incidents, how do they develop experience?AI as a learning partner: How pairing junior engineers with AI agents could create a new way to learn by reviewing and validating AI-generated work.What humans still need to own: Why final sign-offs, domain knowledge, edge cases, and understanding business and customer impact remain critical.The rise of the techno-functional engineer: Why understanding both technology and the business problem may matter more as AI gets better at execution.The economics of AI: Why organizations may need to become more selective about where they use AI as the cost of running increasingly capable models grows.Using AI deliberately: Why the next phase may not be about putting AI everywhere, but figuring out where it actually creates value.AI is evolving quickly, and there is no established playbook for what comes next. As Piyush puts it, we are all learning in real time. The challenge now is figuring out how to embrace AI without losing the skills, judgment, and experience that make great engineers valuable. Listen to Episode 5 of AI-First Engineering to hear where Piyush thinks software engineering is headed next. #AI #AIAgents #SoftwareEngineering #EngineeringLeadership #SDLC #AIFirstEngineering #TestGrid

  5. Aug 19

    What Happens to QA When AI Joins the Team?

    What happens to QA when AI joins the team? In Episode 4 of AI-First Engineering, Paresh Bhutada, Global Partner Manager from TestGrid sits down with Maneesh Agarwal, Director QA at Forsys, to explore how AI is reshaping Quality Engineering, QA roles, and the way modern teams work. Maneesh is a quality engineering leader with extensive experience across manual testing, automation, and AI-driven testing. In this conversation, he shares his perspective on how QA professionals can adapt, where AI can take over repetitive work, and why human judgment, customer experience, and business context will remain critical. The conversation explores: Why traditional QA teams may not be here to stayWhat AI should automate firstWhy human judgment and customer experience still matterHow QA professionals can stay relevantWhat an AI-native Quality Engineering team could look likeWhether 3 AI-enabled engineers could outperform 10 traditional QA engineers 🎙️ What Happens to QA When AI Joins the Team? Host: Paresh Bhutada, Global Partner Manager, TestGrid Guest: Maneesh Agarwal, Director QA, Forsys Watch the full conversation and join the discussion on the future of Quality Engineering. #AIFirstEngineering #QualityEngineering #AITesting #AIEngineering #EngineeringLeadership #TestGrid

  6. Aug 5

    Will AI Make Great Engineers Even Better?

    AI is no longer just helping engineers write code. It's reviewing pull requests, generating tests, documenting systems, and increasingly acting as a collaborative engineering teammate. But as AI takes on more of the software development lifecycle, one question becomes more important than ever: What will distinguish great engineers in an AI-first world? In Episode 3 of AI-First Engineering, Jon Robinson, Head of Market Development at TestGrid, sits down with Suneet Malhotra, Senior IEEE Member and independent practitioner-researcher, to explore how AI is reshaping engineering roles, quality engineering, and software development itself. Together, they discuss why the future belongs to builders, how AI is changing the role of QA, the rise of AI agents and Model Context Protocol (MCP), and why human judgment remains critical in an increasingly autonomous engineering landscape. In this episode: • How AI is making experienced engineers dramatically more productive • Why the future role of QA is evolving into that of a "builder" • How AI agents and MCP are transforming software development workflows • Why human expertise is still essential for validating AI-generated work • The challenges of measuring AI quality and building confidence in AI systems • Whether the QA engineer role will still exist five years from now Guest: Suneet Malhotra is a Senior IEEE Member and independent practitioner-researcher in LLM-augmented software testing, with over 20 years of experience leading quality engineering for consumer-scale platforms including Motorola, Tinder, Amazon, Ticketmaster, the Oprah Winfrey Network, and Spokeo. Host: Jon Robinson, Head of Market Development, TestGrid. AI-First Engineering is a podcast by TestGrid featuring conversations with engineering and technology leaders shaping the future of AI-driven software delivery, quality engineering, and enterprise innovation. Links suneetmalhotra.com github.com/SuneetMalhotra linkedin.com/in/suneet-m

  7. Jul 22

    The Human Side of AI-Driven Engineering

    Everyone is talking about AI. But successful AI transformation is about more than technology. It's about people. In this episode of AI-First Engineering, host Jon Robinson is joined by Peter Thompson to explore the human side of AI-driven engineering. As AI becomes embedded across the software development lifecycle, engineering leaders face new challenges that go beyond tools and automation. How do you build trust in AI? How do leaders adapt? And what role will human judgment, culture, and collaboration play in the future of engineering? Together, they discuss: Why people are often the biggest challenge in AI transformationHow engineering leadership must evolve in an AI-first worldThe importance of culture and trust in AI adoptionWhat successful AI-driven engineering organizations will look likeWhether the future belongs to human skills, technical skills, or bothIf you're an engineering leader, QA professional, software developer, or anyone navigating AI transformation, this episode offers valuable insights into building AI-first organizations where people and technology work together. Guest: Peter Thompson Host: Jon Robinson AI-First Engineering is a podcast by TestGrid featuring conversations with engineering and technology leaders shaping the future of AI-driven software delivery.

    The Human Side of AI-Driven Engineering

  8. Jul 8

    How Do You Test AI That Talks Back?

    How do you test AI that doesn't always behave the same way? As AI-powered applications become more conversational and customer-facing, traditional software testing approaches are no longer enough. Engineering teams must now validate systems that generate non-deterministic responses, learn from interactions, and directly influence customer experiences. In the first episode of AI-First Engineering, Deepanwita Kar, Sr. Director, Partnerships & Alliances at TestGrid, speaks with Chandrashekhar Patil, VP QA at redBus (MakeMyTrip), about the future of AI testing and quality engineering. Together, they explore: Why testing AI systems is fundamentally different from testing traditional software The challenges of validating conversational AI and customer-facing AI applications How engineering teams can build confidence in AI-driven systems The role of trust, user experience, and human behavior in AI validation Practical advice for introducing AI validation into existing testing strategies What quality engineering could look like over the next five years Whether you're an engineering leader, QA professional, AI practitioner, or technology executive, this episode offers valuable insights into building trustworthy AI-powered software. Connect with TestGrid 🌐 Website: https://testgrid.io 💼 LinkedIn: https://www.linkedin.com/company/testgridio

    How Do You Test AI That Talks Back?
  • 8 Episodes

About

The podcast where engineering and technology leaders share how AI is transforming software delivery, quality, and the future of engineering

Information