How I Tested That

David J Bland

Testing your ideas against reality can be challenging. Not everything will go as planned. It’s about keeping an open mind, having a clear hypothesis and running multiple tests to see if you have enough directional evidence to keep going. This is the How I Tested That Podcast, where David J Bland connects with entrepreneurs and innovators who had the courage to test their ideas with real people, in the market, with sometimes surprising results. Join us as we explore the ups and downs of experimentation… together.

  1. 22M AGO

    Chad Holdorf | How I Tested Pull Requests

    Summary In this episode I’m joined by Chad Holdorf, longtime product and technology leader whose career spans John Deere, Salesforce, Pendo, and now Demandbase, where he leads AI initiatives across the company. We explore how AI is fundamentally reshaping the way modern product teams test, ship, and learn, from debugging customer issues directly against live codebases to product managers and support teams submitting pull requests themselves. Chad shares how tools like Cursor and Claude are collapsing traditional handoffs between product, engineering, and support, creating a much faster feedback loop between customer problems, experimentation, and shipped solutions. We also get into the messy reality behind enterprise AI adoption, including data quality, hallucinations, trust, evals, and why testing AI products inside real customer environments is much harder than most demos make it look. Chad gives us a peek into how his own workflow has changed, how his teams are learning by building in real time, and why this moment reminds him of the early days of Lean Startup, where he and I first met. If you’ve been wondering what AI-native product development actually looks and feels like inside a real company, this episode is for you. Takeaways AI is collapsing traditional handoffs between product, engineering, and support teams. Chad described customer support teams going directly into code repositories with AI tools to investigate issues, understand root causes, and eventually submit merge requests themselves. Most enterprise AI demos fall apart when connected to messy real-world customer data. Chad emphasized that “just putting Claude on top of the data” failed quickly without extensive labeling, validation, testing, and human feedback loops. Customers could detect hallucinations within a few prompts. AI systems expose hidden data inconsistencies inside organizations. One example showed AI selecting a custom CRM field that technically produced better targeting results than the field support teams were trained to use, creating confusion about which “truth” was actually correct. Trust has become the critical success factor for enterprise AI adoption. Chad explained that once customers encounter inaccurate outputs, confidence in the system drops immediately, which forces teams to spend enormous time improving prompts, SQL queries, evals, and validation workflows before broader rollout. Product managers are increasingly becoming hands-on builders again. Instead of relying entirely on engineering handoffs, Chad now spends large portions of his week inside Cursor and AI coding agents investigating bugs, generating tickets, reviewing repos, and shaping product direction directly through code conversations. AI-native workflows dramatically compress feedback loops. Problems that previously took days of back-and-forth between support, product, and engineering can now move from customer issue to deployed fix in under an hour through AI-assisted workflows and automated merge requests. The biggest organizational bottleneck is shifting away from engineering speed toward enablement and adaptation. Chad compared this moment to early Agile adoption, where downstream teams like sales, support, and training struggled to keep pace with accelerated shipping cycles. AI is now amplifying that challenge even further. Continuous learning and experimentation matter more than formal process mastery right now. Chad repeatedly compared the current AI moment to the early Agile movement: the people progressing fastest are the ones willing to try tools, build things, stay curious, and learn in public rather than waiting for established best practices or certifications. Guest Links LinkedIn: https://www.linkedin.com/in/chadholdorf/ Demandbase: https://www.demandbase.com/ If your leadership team is about to make a big strategic bet, the real risk usually isn’t the idea, it’s the assumptions behind it that haven’t been surfaced yet. A Decision Sprint is a focused 6–12 week engagement where we extract, map, and test those risks so leaders can make a clear Commit, Correct, or Cut decision before major capital moves. Learn more or apply at precoil.com.

    44 min

  2. APR 29

    Büşra Coşkuner | How I Test B2B vs B2C

    Summary In this episode, I’m joined by Büşra Coşkuner, a product management coach and trainer who helps teams move from project thinking to product thinking. We explore how she approaches product discovery and testing. From building zero-to-one products at Doodle to coaching teams across B2B and B2C environments, Büşra shares how to actually operationalize experimentation beyond just A/B tests. We also dig into how to test when you don’t have much data, how to combine qualitative and quantitative insights, and why many teams get stuck thinking they’re doing product work when they’re really just managing tickets. If you’re trying to build a stronger testing culture or just want to make better decisions this episode will challenge how you think about product metrics and experimentation. Takeaways Product transformation is a leadership decision - If leadership isn’t backing the shift from projects to products, it won’t happen, bottom-up enthusiasm isn’t enough. Most “product orgs” aren’t actually product orgs - Adopting Scrum and calling someone a product owner doesn’t mean you’re doing product, many teams are still just managing tickets. You can’t test what you can’t measure - Without proper data instrumentation, teams fall into a “build, build, build” loop instead of build–measure–learn. Metrics frameworks are a starting point, not the system - Pirate metrics (AARRR) or customer factory models help, but real insight comes from adapting them to your actual business model. Qualitative data is not optional - Quant tells you what is happening, qual tells you why. In low-data environments, qual becomes your primary signal. “No data” is usually an excuse - Even in B2B, you can extract directional insights, from sales teams, customer conversations, and patterns across feedback. A/B testing is over-indexed and often misused - Experimentation goes beyond A/B testing. Many teams default to it even when it’s impractical or irrelevant. Sometimes building is the test - For low-risk features, the fastest way to learn is to ship and observe behavior, treat the release itself as the experiment. B2B testing requires creativity, not scale - From sales-assisted experiments to prototype validation and even WhatsApp groups, testing in small markets is possible if you rethink the approach. AI changes the cost of being wrong - When building becomes cheap, you don’t always need heavy upfront validation, you can test the problem through the solution, as long as you’re willing to kill what doesn’t work. Guest Links LinkedIn: https://www.linkedin.com/in/busra-coskuner/Website: https://busra.co/ If your leadership team is about to make a big strategic bet, the real risk usually isn’t the idea, it’s the assumptions behind it that haven’t been surfaced yet. A Decision Sprint is a focused 6–12 week engagement where we extract, map, and test those risks so leaders can make a clear Commit, Correct, or Cut decision before major capital moves. Learn more or apply at precoil.com.

    51 min

  3. APR 15

    Gil Vaisman | How I Tested ADU’s

    Summary In this episode I’m joined by Gil Vaisman. He’s the founder of GoADU, a construction company focused on building accessory dwelling units. We explore how he went from a 15-year career in film editing to building a construction business that helps homeowners unlock equity and create new living spaces.  What started out as a personal project in his own backyard turned into a growing business built through trial, error, and constant iteration. Gil shares how he tested his way into the market, from helping friends navigate permitting to evolving into a guaranteed fixed pricing model. We also dig into how he qualifies customers, avoids costly mistakes, and thinks about what to test next in an industry that’s rapidly changing. If you’re trying to turn a personal pain point into something scalable, this episode is a great look at how testing can lead to a viable business. Enjoy my conversation with Gil Vaisman. Takeaways Great businesses often start as personal pain points - Gil’s ADU company emerged from building one in his own backyard and helping friends navigate the same confusing process.  Transferable skills matter more than industry experience - His background in film production translated directly into construction, both require coordination, budgeting, timelines, and managing complex teams.  Early traction came from education, not selling - In the beginning, most customers didn’t even know what an ADU was, growth required teaching the market before capturing it.  Trial and error built the real expertise - Navigating difficult permitting processes and making costly mistakes early on became the foundation for a repeatable, refined system.  Pre-qualification is critical in complex services - Gil now asks 20–25 upfront questions before taking on a client, ensuring alignment and reducing downstream risk.  Competing on price is a starting point, not a strategy - The business initially won work by being the cheapest, but evolved into a premium, top-20% offering focused on quality and service.  A strong value proposition can reduce industry fear - Guaranteed fixed pricing became a key differentiator, directly addressing customer anxiety around hidden costs and change orders.  Future innovation is constrained by feasibility, not demand - Customers clearly want faster, cheaper builds (prefab, SIPs), but adoption is limited by execution risk, expertise gaps, and inconsistent quality.  Guest Links GoADU: https://www.goadu.com/ Vaisman Construction: https://www.vaismanconstruction.com/ If your leadership team is about to make a big strategic bet, the real risk usually isn’t the idea, it’s the assumptions behind it that haven’t been surfaced yet. A Decision Sprint is a focused 6–12 week engagement where we extract, map, and test those risks so leaders can make a clear Commit, Correct, or Cut decision before major capital moves. Learn more or apply at precoil.com.

    31 min

  4. APR 1

    Bill Fienup | How I Tested a HardTech Innovation Center

    In this episode I’m joined by Bill Fienup. He’s the co-founder of mHUB, one of the world’s leading hardtech innovation centers, located in Chicago, IL. We explore how he went from building Nerf gun prototypes at MIT to creating a space where thousands of hardware founders can prototype, test, and scale physical products. What started out as a meetup group and a spreadsheet, grew into a full ecosystem with millions of dollars in equipment and billions of dollars in economic impact. Bill shares how to test hardware ideas without burning capital, why most teams over-focus on feasibility instead of desirability, and how to validate what people will actually pay for before you build. If you’re working on physical products, or funding them, this episode is a masterclass in how to test before you invest. Enjoy my conversation with Bill Fienup. Takeaways Start with the problem, not the solution. The biggest risk isn’t building something, it’s solving a problem that customers don’t care enough about to act on. Desirability and willingness to pay matter more than feasibility early. Teams often over-focus on building, but the real uncertainty is whether customers value the solution enough to pay. Test demand before investing in development. Simple experiments like landing pages or fake purchase flows can validate real interest before committing resources. Iterate in spirals, not stages. Move across desirability, feasibility, and viability repeatedly, increasing investment only as uncertainty is reduced. Avoid building the wrong thing the right way. Strong execution can’t fix a fundamentally misaligned product, validation must come before scale. Use competition as validation. Existing solutions signal real demand and confirm the problem is worth solving. Focus on the majority, not edge cases. Designing for the loud minority can increase cost and complexity without improving overall product-market fit. Community can be a powerful starting point. MHub began as a meetup and shared spreadsheet, showing how real user pain can evolve into a scalable ecosystem. Guest Links mHub’s Website: https://www.mhubchicago.com/ LinkedIn Profile: https://www.linkedin.com/in/fienup/ If your leadership team is about to make a big strategic bet, the real risk usually isn’t the idea, it’s the assumptions behind it that haven’t been surfaced yet. A Decision Sprint is a focused 6–12 week engagement where we extract, map, and test those risks so leaders can make a clear Commit, Correct, or Cut decision before major capital moves. Learn more or apply at precoil.com.

    35 min

  5. MAR 18

    Akvile Ignotaite | How I Tested a TikTok Pimple

    Summary In this episode I’m joined by Dr. Akvile Ignotaite, a data scientist and founder building AI-powered skin health technology used by more than 800,000 people around the world. We explore how her team combines data science, health tech, and creative marketing to rethink skincare for Gen Z and Gen Alpha. From building a vast skin care dataset to launching a TikTok influencer pimple called Pimsy that has almost 40k followers, Akvile shares how cultural insights and small tests drive their product strategy. We also get into the challenges of building health technology for younger audiences, how to test ideas across different global markets, and why treating skin as a health problem, changes how you design products and measure success. If you’re interested in experimentation and AI in health you’ll enjoy my chat with Akvile. Takeaways Start small and imperfect to learn faster. The team prioritizes quick MVPs, sometimes built in days, to test ideas before investing heavily in development, branding, or marketing. Customer language and psychology matter. The original millennial-focused “compliance app” failed because it sounded too technical; shifting to Gen Z language, emojis, and storytelling dramatically improved adoption. Meet users where they already are. Channels like TikTok became critical for reaching younger audiences, even though the team initially resisted the platform. Creative experimentation can unlock growth. The “Pimsy” influencer pimple character started as a small test and quickly grew to tens of thousands of followers, proving unconventional ideas can resonate strongly with audiences. Micro-learning can drive high engagement. A simple, quickly built “myths vs. facts” quiz feature created massive engagement and generated valuable behavioral data about user beliefs. User feedback is a competitive advantage. Hiring a developer who criticized the Android experience highlighted the importance of listening closely to real user complaints and improving where customers actually are. Cultural assumptions can mislead founders. Expanding into India revealed how preconceived ideas about markets, healthcare practices, and culture can be wrong, reinforcing the need for curiosity and humility. Structured programs don’t always fit real user behavior. Highly designed 6- or 8-week skincare programs failed because users resisted rigid routines, showing how human behavior often breaks logical product design. Gen Z and Gen Alpha are forming a global digital culture. The app’s success without localization suggests younger generations increasingly share common digital behaviors and language across regions. Guest Links System Akvile: https://systemakvile.com/LinkedIn Profile: https://www.linkedin.com/in/dr-akvile-ignotaite/Pimsy TikTok: https://www.tiktok.com/@i.am.pimsy If your leadership team is about to make a big strategic bet, the real risk usually isn’t the idea, it’s the assumptions behind it that haven’t been surfaced yet. A Decision Sprint is a focused 6–12 week engagement where we extract, map, and test those risks so leaders can make a clear Commit, Correct, or Cut decision before major capital moves. Learn more or apply at precoil.com.

    36 min

  6. MAR 4

    Jim Morris | How I Test My Teaching Process

    In this episode I’m joined by Jim Morris.  We chat about the wake-up call that pushed him from building first to testing first. Jim and I discuss loyalty programs no one wanted, roadmaps filled with sequenced risk, AI prototypes that hallucinate and the uncomfortable reality that confidence often replaces evidence. We also dig into something deeper: why smart teams ignore data, why leaders fall in love once an idea hits the roadmap, and why testing isn’t about better UX,  it’s about real value. Jim shares how he even tests his own teaching process for students at Berkeley. Because as he puts it: “We can build stuff. But if people don’t use it, we’re just creating product debt.”Enjoy my conversation with Jim Morris. Takeaways Testing is crucial to ensure product effectiveness and user engagement. Data analysis can reveal the true usage of product features. Mindset plays a significant role in how product ideas are perceived and developed. Not all ideas will succeed; testing helps identify the viable ones. User motivation is key to the success of features and programs. Prototyping tools can enhance the testing process but require careful implementation. Learning from failures in testing is essential for growth and improvement. Roadmaps should be flexible to adapt to changing priorities and evidence. It's important to focus on the core value proposition of a product. Continuous experimentation and adaptation are vital in product management. Guest Links Website: https://productdiscoverygroup.com/ LinkedIn Profile: https://www.linkedin.com/in/jimmorrisstanford/ If your leadership team is about to make a big strategic bet, the real risk usually isn’t the idea, it’s the assumptions behind it that haven’t been surfaced yet. A Decision Sprint is a focused 6–12 week engagement where we extract, map, and test those risks so leaders can make a clear Commit, Correct, or Cut decision before major capital moves. Learn more or apply at precoil.com.

    39 min

  7. FEB 18

    Dan Olsen | How I Test With Vibe Coding

    Summary In this conversation, David J Bland and Dan Olsen discuss the evolution of product management, the impact of vibe coding, and the importance of cross-functional collaboration. They explore the challenges of prototyping, user research, and the role of AI in product development. The discussion emphasizes the need for strong product management fundamentals and the future of product management in a rapidly changing landscape. Takeaways The awareness of product management has significantly increased over the years.Vibe coding allows for rapid prototyping and testing without heavy technical resources.Cross-functional collaboration is essential for successful product development.User research is becoming more valued in product management.Prototyping should focus on learning rather than just building.AI can assist in generating ideas but lacks judgment in prioritization.The pace of innovation in product tools is accelerating rapidly.Understanding customer problems is crucial for product success.Rushing to high fidelity prototypes can lead to missed opportunities in the problem space.Product management fundamentals will be key in differentiating successful products. Guest Links Website: https://dan-olsen.com/ LinkedIn Profile: https://www.linkedin.com/in/danolsen98/ YouTube Channel: https://www.youtube.com/danolsen Lean Product Meetup: https://www.meetup.com/lean-product/  Vibe Coding Product Brief: https://dan-olsen.com/vibe-coding/ Vibe Coding Spectrum: https://dan-olsen.com/vibe-coding/ The Lean Product Playbook: https://amzn.to/1EYCUdP If your leadership team is about to make a big strategic bet, the real risk usually isn’t the idea, it’s the assumptions behind it that haven’t been surfaced yet. A Decision Sprint is a focused 6–12 week engagement where we extract, map, and test those risks so leaders can make a clear Commit, Correct, or Cut decision before major capital moves. Learn more or apply at precoil.com.

    55 min

  8. 12/10/2025

    David Sauers | How I Tested Royal Restrooms

    Summary In this episode we interview David Sauers. He’s the founder of Royal Restrooms, a company that’s redefining what a portable restroom can be.  What started with a frustrating moment at a festival turned into David challenging a fundamental assumption: How can we elevate one of the most overlooked parts of any event? He walks us through why weddings became their breakthrough customer segment and how adapting designs for different event types unlocked growth. This isn’t just about restrooms,  it’s about challenging stigma, listening closely to customer signals, and innovating in overlooked spaces.  David explains how they are testing their way into showers, beverage trailers, and beyond. Guest Links LinkedIn Profile: https://www.linkedin.com/in/davidsauers/ Royal Restrooms: https://royalrestroomsca.com/ Personal Website: https://www.davidsauers.com/ If your leadership team is about to make a big strategic bet, the real risk usually isn’t the idea, it’s the assumptions behind it that haven’t been surfaced yet. A Decision Sprint is a focused 6–12 week engagement where we extract, map, and test those risks so leaders can make a clear Commit, Correct, or Cut decision before major capital moves. Learn more or apply at precoil.com.

    28 min
See All (48)

Ratings & Reviews

5
out of 5
4 Ratings

About

Testing your ideas against reality can be challenging. Not everything will go as planned. It’s about keeping an open mind, having a clear hypothesis and running multiple tests to see if you have enough directional evidence to keep going. This is the How I Tested That Podcast, where David J Bland connects with entrepreneurs and innovators who had the courage to test their ideas with real people, in the market, with sometimes surprising results. Join us as we explore the ups and downs of experimentation… together.

Information

  • Creator
    David J Bland
  • Years Active
    2024 - 2026
  • Episodes
    48
  • Rating
    Clean
  • Copyright
    © 2024 Precoil, LLC
  • Show Website
    How I Tested That

You Might Also Like