Behind the Craft

Peter Yang
  • 4.6 (11)
  • Technology
  • Updated Weekly

Expert interviews and guides to help you level up as a product leader and creator fast.

  1. 5d ago

    How to Build Better AI Evals with Claude Code in 5 Steps | Shreya & Hamel

    📌 Copy the AI system I use to run a one-person $1M+ business: https://www.behindthecraft.com 🤝 Subscribe to my practical AI newsletter read by 150K+ professionals: https://creatoreconomy.so Shreya and Hamel teach the industry standard course for AI evaluations taken by 4,500+ students from OpenAI, Google, and more. In our interview, I asked them to give live feedback on the AI evals that I built for my skills and demo how anyone can run evals with ChatGPT/Claude and their free evals skill. Don’t miss this up to date primer on evals from the real experts. 🙂 P.S., I asked Shreya and Hamel to give my audience 25% off the next cohort of their AI evals course. It has 4.7 stars with 900 reviews which is almost unheard of on Maven. Sign up via the link here: https://maven.com/parlance-labs/evals?utm_campaign=peteryang&utm_medium=affiliate&utm_source=maven&promoCode=PETERYANG We talked about: (00:00) Why AI evals are completely different now with the latest models (02:30) Demo: Live audit of the evals that I built for my AI skills (05:14) Top down vs. bottom up evals and why AI sucks at the latter (08:16) Spinning up AI agents to grade each eval criteria (14:32) Demo: Using Shreyas AI skill to run evals in Claude (23:46) How to review output labels to identify failures (34:37) How to turn failures into reusable evals (43:19) Where human judgement is still needed Resources mentioned: Free eval skills from Shreya: https://github.com/ai-evals-course/evals-skills Do automated evals work?: https://parlance-labs.com/blog/posts/auto-evals/ Thanks to our sponsors: Wispr Flow — 4x faster than typing with your voice — https://ref.wisprflow.ai/peteryang 📌 Get the takeaways: https://creatoreconomy.so/p/how-to-build-better-ai-evals-with-claude-code-shreya-hamel Where to find Shreya and Hamel: Shreya: https://www.sh-reya.com/ Hamel: https://hamel.dev/ Subscribe to this channel - more interviews coming soon!

  2. Aug 16

    How I Run My 1.5M+ Follower Content Business With Codex | Riley Brown

    📌 Copy the AI system I use to run a one-person $1M+ business: https://www.behindthecraft.com 🤝 Subscribe to my practical AI newsletter read by 150K+ professionals: https://creatoreconomy.so Riley is a top AI creator and founder of Agent Native. In this episode, he showed me how he uses Codex to run his entire content business from generating engaging thumbnails to writing incredible video hooks. Riley also shared his full AI stack including some lesser known AI tools to edit videos, draw diagrams, and more. We talked about: (00:00) How Codex runs Riley’s 1.5M+ follower content business (03:16) Demo: Using Codex to create animations and B-roll (07:28) Demo: Codex can reverse-engineer a top YouTube hook (13:24) The BRENS framework to turn an idea into a strong hook (17:24) Demo: Riley’s Codex stack for diagrams, thumbnails, and posts (22:01) How Riley creates so much content every week (25:41) Demo: Using Codex and Paper for viral YouTube thumbnails (35:01) How to set up AI agents to find content ideas proactively Thanks to our sponsors: Google AI Studio: The fastest way to go from prompt to production https://aistudio.google.com/?utm_source=peteryang&utm_medium=partner&utm_campaign=26Q3-peteryang_ais Riverside: All-in-one AI studio for podcasts and video https://creators.riverside.com/PeterYang Wispr Flow: 4x faster than typing with your voice https://ref.wisprflow.ai/peteryang 📌 Get the takeaways: https://creatoreconomy.so/p/how-i-run-my-content-business-with-codex-riley-brown Where to find Riley: Agent Native, Riley’s new company: https://agentnative.inc/ X: https://x.com/rileybrown Website: https://rileybrown.xyz/ Subscribe to this channel - more interviews coming soon!

  3. Aug 9

    5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

    📌 Copy the AI system I use to run a one-person $1M+ business: https://www.behindthecraft.com 🤝 Subscribe to my practical AI newsletter read by 150K+ professionals: https://creatoreconomy.so Nan and Jacob built Linear Agent, which has transformed how the $1.25B company gets work done. In this episode, they took me from the initial memo to launch, showing how to give an agent tools to find the context it needs, use evals to improve reliability, and more. This episode is a must-watch if you want a concrete, behind-the-scenes example of how to build a production agent end to end. We talked about: (00:00) What’s holding AI agents back as models improve? (00:51) Three things every good agent needs (03:00) Demo: The internal memo that started Linear Agent (09:50) The biggest mistake to avoid when building agents (11:29) Demo: From Slack to code in 6 minutes with the agent (17:01) When prototyping, “use the biggest model until it works” (24:02) Give agents tools to find context, not more context (34:07) The first thing to do before building an agent Thanks to our sponsors: Oceans: Hire AI-native EAs http://oceanstalent.com/peter Riverside: All-in-one AI studio for podcasts and video https://creators.riverside.com/PeterYang Linear: The AI agent platform for modern teams https://linear.app/behind-the-craft 📌 Get the takeaways: https://creatoreconomy.so/p/5-rules-for-building-ai-agents-in-production-linear-nan-jacob Where to find Nan and Jacob: Nan on LinkedIn: https://www.linkedin.com/in/thenanyu Jacob on LinkedIn: https://www.linkedin.com/in/jacob-shumway-314a8366 Linear Agent: https://linear.app/docs/linear-agent Subscribe to this channel - more interviews coming soon!

  4. Aug 2

    Hermes Co-Founder on Building an AI Agent That Improves Itself | Karan Malhotra

    📌 Copy my personal AI system to save time and build with AI: https://www.behindthecraft.com/ 🤝 Get my free AI newsletter read by 150K+ professionals: https://creatoreconomy.so/ Karan co-founded Nous Research, the company behind Hermes, the #1 personal agent and AI app overall on OpenRouter. I found him incredibly down to earth as we discussed why open-source AI must win, how Hermes gets better the more you use it, and even a live demo of how Hermes can mod your favorite childhood games. Karan and I talked about: (00:00) How Hermes differs from Claude Code and Codex (02:20) Why every AI model now sounds exactly the same (11:16) How to stop AI from saying “you’re absolutely right” (15:16) How Hermes creates skills without bloating its context (16:40) The open vs. closed-source debate on AI safety (23:40) Using one Kanban board for humans and AI agents (26:45) Demo: Rebuilding a childhood Sonic world with Hermes (36:07) How two Discord friends built a self-improving agent (42:40) How Hermes became its own biggest contributor Thanks to our sponsors: Riverside: All-in-one AI studio for podcasts and video https://creators.riverside.com/PeterYang Linear: The AI agent platform for modern teams https://linear.app/behind-the-craft Wispr Flow: 4x faster than typing with your voice https://ref.wisprflow.ai/peteryang 📌 Get the takeaways: https://creatoreconomy.so/p/hermes-co-founder-on-building-an-ai-agent-that-improves-itself-karan-malhotra Where to find Karan: X: https://x.com/karan4d Website: https://www.karan4d.com/ Hermes Agent: https://github.com/NousResearch/hermes-agent Subscribe to this channel - more interviews coming soon!

  5. Jul 26

    ChatGPT Work + Codex Tutorial: My Complete System at OpenAI | Jason Liu

    Jason is a DevEx engineer on the Codex team. In our episode, he showed me how he runs his entire workday using Codex, from his chief of staff thread to the way he builds skills that learn how he works. Jason wrote the official OpenAI guidebook on how to get the most out of Codex, so don’t miss this episode. Jason and I talked about: (00:00) The only job left after AI can do the work (02:40) Turn pinned threads into long-running workspaces (05:50) Build a chief of staff across Slack, email, and Linear (10:12) Teach Codex to write like you for every audience (13:33) Mine past sessions for new skills and workflows (20:01) How Codex revised a launch video while Jason was out (24:23) Build a drum practice site around how you learn (25:59) Goal, plan, and worklog for long-running Codex tasks (33:52) Why taste starts with knowing what feels wrong Thanks to our sponsor: Google AI Studio: The fastest way to go from prompt to production https://aistudio.google.com/?utm_source=peteryang&utm_medium=partner&utm_campaign=26Q3-peteryang_ais Riverside: All-in-one AI studio for podcasts and video https://creators.riverside.com/PeterYang 📌 Get my personal AI OS with useful AI skills, $270 in AI tool credits, live monthly workshops, and courses on Hermes and Codex agents: https://www.behindthecraft.com/ Where to find Jason: X: https://x.com/jxnlco Subscribe to this channel - more interviews coming soon!

  6. Jul 19

    How I Plan, Build, and Run Loops with Claude Code in 40 Minutes | Thariq Shihipar

    Thariq works on the Claude Code team, and I’ve wanted to see how he builds for a long time. In our episode, he showed how to use /goal to keep Claude working, how he plans with Claude to remove unknowns before building, and how he runs a team of agents in Slack. He also shared why his team cut Claude Code’s system prompt by 80% and how he creates polished launch videos with Claude. Thariq and I talked about: (00:00) Goals, loops, and workflows to make Claude work longer (04:48) Live demo: Creating a polished video with one prompt (08:17) Plan by removing unknowns before you build (14:32) How Anthropic uses HTML artifacts for planning (18:35) Running parallel Claude tasks through Slack (27:34) Using separate creator and verifier sub-agents (31:08) How Thariq stays focused while Claude works in parallel (34:07) Why Anthropic cut Claude Code’s system prompt by 80% (37:01) How to become more technical when AI writes the code Thanks to our sponsors: Riverside: All-in-one AI studio for podcasts and video https://creators.riverside.com/PeterYang Linear: The AI agent platform for modern teams https://linear.app/behind-the-craft Wispr Flow: 4x faster than typing with your voice https://ref.wisprflow.ai/peteryang 📌 Get the takeaways: https://creatoreconomy.so/p/how-i-plan-build-and-run-loops-with-claude-code-thariq-shihipar 📌 Get my personal AI OS with useful AI skills, $270 in AI tool credits, live monthly workshops, and courses on Hermes and Codex agents: https://www.behindthecraft.com/ Where to find Thariq: LinkedIn: https://x.com/trq212 Website: https://www.thariq.io/ Subscribe to this channel - more interviews coming soon!

  7. Jul 12

    How to Build AI Agents That Check Their Own Work | Jared Zoneraich

    Jared is a Builder in Residence at Cognition, the team behind Devin. He showed me why teams often overbuild their first agents, and how simple prompts, better tools, and self-testing work together instead. Then he demoed one manager agent breaking down a task, directing 10 cloud agents, and deciding what a human needs to review. Jared and I talked about: (00:00) Why teams overbuild their first AI agent (02:50) Let the model cook before adding rules (03:16) Why tools matter more than better prompts (10:51) Why you should build evals after shipping (16:06) What cloud agents can do that local agents can’t (20:30) How to get one agent to manage other agents (25:39) Demo: One agent launches a team of 10 (28:03) How agents prove their work before you review it (33:18) The best tasks for agent teams to work on Thanks to our sponsors: Riverside: All-in-one AI studio for podcasts and video https://creators.riverside.com/PeterYang Linear: The AI agent platform for modern teams https://linear.app/behind-the-craft Wispr Flow: 4x faster than typing with your voice https://ref.wisprflow.ai/peteryang 📌 Get the takeaways: https://creatoreconomy.so/p/how-to-build-ai-agents-that-check-their-own-work-jared-cognition-devin 📌 Get my personal AI OS with useful AI skills, $270 in AI tool credits, live monthly workshops, and courses on Hermes and Codex agents: https://www.behindthecraft.com/ Where to find Jared: X: https://x.com/imjaredz Devin: https://devin.ai Subscribe to this channel - more interviews coming soon!

  8. Jul 5

    OpenAI PM Reveals How He Uses Codex to Do Product Work | Rohan Varma

    As a product manager for OpenAI Codex, Rohan has a firsthand look at how one of the most AI-native teams in the world operates. He showed me how he uses Codex to trigger work from Slack replies, prototype designs with image generation, replace stale PRDs with live project sites, and more. Watch this episode if you’re tired of all the nonsense around “AI-native PM” and want to see how a real one actually operates. Rohan and I talked about: (00:00) Codex can do more than you think (01:53) Use Codex to pull context from Slack, docs, Linear, and email (03:28) How Codex flips planning from docs to demos (13:14) Demo: Do the task once with Codex, then automate it (14:30) Demo: Trigger Codex automations from Slack replies (17:26) Demo: Use Image Gen to explore design variants fast (19:58) Demo: Turn any Codex thread into a reusable skill (29:55) Demo: Turn project context into a shareable Site (33:33) Use one Codex thread to manage other Codex threads (37:18) Aim 10x more ambitious than feels reasonable Thanks to our sponsors: Linear: The AI agent platform for modern teams https://linear.app/behind-the-craft Wispr Flow: 4x faster than typing with your voice https://ref.wisprflow.ai/peteryang Riverside: All-in-one AI studio for podcasts and video https://creators.riverside.com/PeterYang 📌 Get the takeaways: https://creatoreconomy.so/p/openai-pm-reveals-how-he-uses-codex-for-product-work-rohan-varma 📌 Get my personal AI OS with useful AI skills, $270 in AI tool credits, live monthly workshops, and courses on Hermes and Codex agents: https://www.behindthecraft.com/ Where to find Rohan: LinkedIn: https://www.linkedin.com/in/rohannvarma X: https://x.com/TheRohanVarma Codex: https://openai.com/codex/ Subscribe to this channel - more interviews coming soon!

See All (121)

Ratings & Reviews

4.6
out of 5
11 Ratings

About

Expert interviews and guides to help you level up as a product leader and creator fast.

Information

  • Creator
    Peter Yang
  • Years Active
    2024 - 2026
  • Episodes
    121
  • Rating
    Explicit
  • Copyright
    © Peter Yang
  • Show Website
    Behind the Craft

You Might Also Like