Building AI Products

What you’ll learn in this episode:

• The difference between an AI-powered product manager and an AI product manager
• Why prompt engineering for a product is different from prompting ChatGPT for personal use
• The role of prompt decomposition and orchestration in building robust AI features
• How to think about system design, risk mitigation, and cross-functional collaboration
• Why observability and logging traces are critical for LLM products
• The challenge of evaluating non-deterministic AI features (and why “thumbs up/thumbs down” isn’t enough)
• How to decide when AI is the right solution for a customer problem
• The hidden cost of ongoing maintenance for AI features

Resources & Links:

• Follow Teresa Torres: https://ProductTalk.org
• Follow Petra Wille: https://Petra-Wille.com

Mentioned in this episode:

• Behind the Scenes: Building the Product Talk Interview Coach by Teresa
• How I Designed & Implemented Evals for Product Talk’s Interview Coach by Teresa
• Leading Through the AI Wave—With Clarity, Not Confusion by Petra Wille
• AI Prototyping - All Things Product with Teresa & Petra episode
• ChatGPT
• Aman Khan — distinction between AI-powered vs. AI product managers
• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO) - Lenny’s Newsletter
• Devin AI and Windsurf (acquired by Cognizant)
• Yelp
• Uber
• Large language model (LLM)
• Claude
• Gemini
• BlackBerry
• Replit
• Zapier
• Amazon Web Server (AWS) Step Functions
• AI Agents in Practice: How Henrik Kniberg Sees the Future of Collaborative Work - Henrik Kniberg’s “interns” metaphor from Product at Heart 2025
• Anthropic
• Extensible Markup Language (XML)
• OpenAI
• Markdown
• Eval (short for evaluate)
• Personal information or personally identifiable information (PII)
• Machine learning (ML)