Live with Tim O’Reilly

O'Reilly

A series of candid conversations designed to slow down, dig deeper, and share real insights you can build on. Get the story behind the show here: https://www.oreilly.com/radar/more-slowly/

  1. 1 day ago

    Escaping the Prompt Debt Trap: Drew Breunig Live with Tim O’Reilly

    Drew Breunig has spent years watching how AI teams grow their LLM applications, and he's identified a problem most teams don't notice until it's already slowing them down. As CEO of CMPND, a maintainer of the DSPy open source framework, and author of the forthcoming O'Reilly publication Context Engineering Handbook, he's seen the same pattern play out over and over. A team builds a quick LLM prototype, it mostly works, and then they start patching the gaps, one edge case at a time, one instruction at a time, until the prompt is so bloated and fragile that nobody wants to touch it, it stays pinned to a model from two years ago, and the person who built it is the only one who understands it. If that sounds familiar, you're probably already dealing with what Drew calls prompt debt.Watch as he joins Tim O'Reilly to explain why it's more widespread than most teams realize, and what the best builders are doing instead. Drew traced prompt debt back to two root causes. The first is that natural language is a fundamentally imprecise programming medium. Studies show the same medical question phrased as a patient versus a psychiatrist yields completely different answers, and in one example from the talk, claiming to be a Philadelphia Eagles fan was enough to bypass model safety guardrails entirely. The second is that the biggest labs are increasingly tuning models like Claude for their own products in ways that quietly work against anyone building something custom on top. The conversation also covers why this reliability-over-diversity trade-off is making frontier models worse writers even as they improve at code, why open-weight models may be the Apache to OpenAI and Anthropic's Netscape, and what you can do now, from investing in evals and decomposing tasks to treating prompts as perishable, to build AI systems that hold up. As Drew puts it, "Vibe coding makes code free, as in puppies. You can go get a puppy for free, but now you have to take care of it and it's in your house."

  2. 6 days ago

    A Playbook for Going AI Native: Dan Guido Live with Tim O’Reilly

    Trail of Bits is a software security research and development firm that helps clients like Facebook and DARPA solve their hardest security challenges. In 2024 cofounder and CEO Dan Guido decided the company would go fully AI native. Most of the firm's roughly 130 employees didn't buy in right away, but as of today AI is a “core participant” not just in engineering work but across the full consulting lifecycle. (Sales and operations folks are even using GitHub everyday and contributing to the company skills repository.) Dan joined Tim for an encore of a talk he gave at the [un]prompted conference in March laying out the playbook that made this transformation possible, which Tim called “far and away the best framework and discipline for going AI native." Dan then stuck around for a Q&A to flesh out some of those ideas with the live audience.Most companies get stuck at using AI as a tool rather than turning it into an operating system that compounds expertise across the entire organization because people resist adopting new technologies. For AI specifically, Dan lays the blame on a quartet of very understandable psychological barriers: a bias toward trusting your own (human) judgment over a machine's, an identity threat that makes augmentation feel like replacement, the opacity of not knowing how an AI reached a conclusion, and a low tolerance for AI's mistakes. Trail of Bits built its transformation around those biases instead of against them by standardizing on a single toolset (Claude Code), writing an internal AI handbook, creating a maturity matrix that made AI skill a normal part of career growth, running hackathons instead of mandates, and capturing everything the company learned into internal, external, and curated skills repositories. The approach has produced real results. Dan said roughly one in five bugs the firm now finds for clients is first surfaced by AI, and its go-to-market team is outperforming typical consulting industry benchmarks.That’s not to say that Trail of Bits’ transformation is complete. Asked what still doesn't work, Dan pointed to privacy-preserving inference and to the lack of any real feedback loop for learning from how employees actually use the tools day-to-day, both of which he called open problems the industry hasn't solved yet. However, the playbook the company has developed, alongside all the skills and other tools it’s shared publicly, will get you a long way toward transforming your own company. As Dan emphasized, “This is not a strategy deck. This is what we actually did.”

  3. 28 July

    The Days When Fable Got Hit by a Bus and Disappeared with Gene Kim

    Long-running Claude Fable 5 agents had become the backbone of Gene Kim's personal knowledge system, one built from thousands of screenshots, videos, and half-finished side projects, spanning 50 code repositories and tens of thousands of changes. He’d been warned that Fable might be pulled from his subscription and got to preparing for that eventuality. But it disappeared eight days earlier than he planned for, mid-session, when Anthropic suspended access over an export-control issue. As he put it, the ship was soon “on fire, tumbling in space.” Gene’s the coauthor of The Phoenix Project, The DevOps Handbook, Accelerate, and, most recently, Vibe Coding (with previous guest Steve Yegge) as well as founder of the Enterprise AI Summit, so he knows a thing or two about dealing with unplanned disasters. He joined Tim for an encore of the lightning talk he gave at O'Reilly's Foo Camp about how he recovered his system, followed by a Q&A on what the shutdown taught him. Gene described watching his systems fail once the less-capable Claude Opus took over from Fable mid-session. Commands that should have worked started breaking, the model argued with itself over which instance had done what, and every action risked making things worse. It took roughly three hours that night, and more the next morning, to stabilize things, largely by recovering documentation Fable had written along the way, and Gene took the audience through the process step-by-step. He and Tim also discussed the deeper lesson of dependency risk from AI providers. We hear a lot about pricing and availability changes, but export controls, shifting safety policy, and geopolitics can force the same kind of sudden cutoff. Gene helped us understand why the solution is the same preventive, rehearsed resilience DevOps engineers built over the last 15 years for large-scale outages, now applied to individuals running mission-critical agents of their own.

  4. 27 July

    Your Next Product Is a Process with Harper Reed

    Harper Reed has spent his career building software people actually love working on, from cofounding Threadless to leading engineering for the Obama 2012 campaign. He joined Tim for a conversation about what happens once software becomes less a product and more a medium for expressing values, and why that shift has made what Harper calls "conviction collapse" (the point where a product pipeline moves so fast there's no time to actually fall in love with what you're building) a real risk. With 2389 Research, the company he runs with his partner, Dylan Richard, Harper has been answering that by treating the product itself as optional, letting an accumulation of small, working experiments define what the company becomes, an approach he compares to the old Unix philosophy of small programs built to scratch a specific itch. Some of those experiments got playful. 2389 built an internal social network where its coding agents post updates, pick names, and develop personalities (Harper's own agent goes by Doctor Biz), and early evals suggested letting agents have fun barely hurt task performance while making the work far more enjoyable to watch. On the cost side, Harper described Thrifty, a skill 2389 just released that pairs a strong model for planning with a cheaper model for execution, cutting token costs by roughly 64% compared to Opus at equal quality. But underneath all the experimentation runs a deep humanism: How do we make sure that everybody—workers and users, privileged and not-so-privileged alike—benefit from transformations AI promises (or threatens, depending on your outlook)? It’s a question animating everything from AI tool adoption to the values an organization professes to society writ large. "I want to make sure the beauty and fun I had over my career doing technology...maintains," Harper explained, "because that's the reason I'm here."

  5. 11 June

    Making AI Relatable: Harper Carroll Live with Tim O’Reilly

    Harper Carroll is a computer scientist who built machine learning systems at Meta but she also describes herself as "born an actress from Manhattan." She’s combined those disparate parts of her background into a unique role as an AI educator, where she uses her magic superpower of making sense of AI to reach half a million people on social media and beyond. She has a knack for explaining how models actually work, covering concepts like optimization and token distributions and the math behind them in terms that land for people who've never opened a Python notebook. Harper sat down with Tim to talk about how she makes technical complexity incredibly relatable, but they also thought through some of the more comprehensive challenges the industry is facing. Those ranged from the technical, as Harper explained why fine-tuning a small open source model beats prompting even the best closed-source model when you're trying to capture voice, to cultural considerations like the need to shift the narrative from fearing AI to explaining how AI can expand ambition both for individuals and for organizations, why we should treat AI as a medium like photography or writing, and why open source AI is a much bigger story than open source models. And in keeping with both Harper’s and Tim’s focus on learning, they discussed the skills everyone in the workforce will need to have to use AI effectively. That’s a social problem to the extent that we’ll need to ensure that everybody learns enough about AI so we don't end up with AI haves and have-nots. But it’s also a recognition that AI education is becoming a critical part of the path to success for all kinds of jobs. "The people who are really going to struggle," Harper told Tim, "are the people who are not willing to accept that AI is coming and are not willing to learn it."

    Making AI Relatable: Harper Carroll Live with Tim O’Reilly
  6. 3 June

    Data Access and Other Bottlenecks in Enterprise AI Adoption: DJ Patil Live with Tim O’Reilly

    DJ Patil co-coined the term "data scientist," served as America's first chief data scientist under President Obama, was chief scientist at LinkedIn, and has spent the past decade on the founding team at Devoted Health, where he's built the kind of data infrastructure that most organizations are still struggling to create. DJ’s been on a listening tour. Wherever he travels, he finds a local university, holds office hours, and asks whoever shows up—students, faculty, hospital administrators, executives—what they're actually experiencing with AI. What he's hearing about the anger, angst, and the “impasse of dialogue” between AI boosters and skeptics is helping him refocus his thoughts about AI adoption. He joined Tim for a wide-ranging conversation about what's working right now, what's broken, and where the real bottlenecks are. DJ and Tim started with students, who are feeling that the social contract of “go to college and start a career” is broken, and DJ's plan to launch a makerspace-style program for those who didn’t land internships this summer to learn and demonstrate their skills. Then they went deep on why the organization is the bottleneck to transformation, using the healthcare industry as an example of the entrenched challenges and what’s possible when you get the infrastructure right. DJ walked through how Devoted Health built its data foundation before LLMs existed, why that tidy house is now a compounding advantage, and the change we can make when transforming our healthcare system is, as he put it, "like walking, chewing gum while balancing bowling balls on your head and on a unicycle." "We're [both] this giant human LLM," DJ told Tim, "summarizing and distilling what we're hearing from a lot of people." What they're hearing is that the chief constraint is whether our institutions can build the organizational and economic infrastructure to actually deploy what we've built.

  7. 27 May

    Inside the Code Factory: Ryan Carson Live with Tim O’Reilly

    Ryan Carson has spent 25 years building developer communities, conferences, and Treehouse, which taught over a million people to code. His latest company, Untangle, is an AI-powered divorce assistant—and he’s building it entirely alone. Just $2 million in seed funding, his guidance, and a team of agents running while he sleeps. Ryan sat down with Tim to walk through the “code factory” powering Untangle: a system where agents write and review the code, run the tests, triage error reports, and monitor the production environment under his oversight. In their conversation, they covered the Ralph Wiggum loop (Geoffrey Huntley’s deceptively simple technique for giving agents large goals across multiple context windows) and the power of primitive loops, how Ryan used Claude Design and a human designer to build a full design system he can now reproduce with AI, what attorneys really think about Untangle, the economics of running a company of agents, why the narrative that programming is going away gets the abstraction story exactly backwards, and why, even when you can automate nearly everything else, you still can’t automate the judgment call about what to build. “There isn’t a magic wand still,” Ryan told Tim. “You can build faster, but whether you’re building the right thing, and doing it better, is something [else].” Read Tim’s takeaways from the conversation, plus clips, on Radar.

  8. 26 May

    A Conversation with Computer Programmer Steve Yegge

    Longtime software engineer Steve Yegge has lately been exploring the limits of vibe coding with projects like Beads, his coding agent memory system, and Gas Town, a proof-of-concept agent orchestrator so complicated, expensive, and chaotic that he warned those interested NOT to use it. As Tim O’Reilly points out in his takeaways from this episode, “Steve has always been one of the most provocative thinkers in our industry.” Steve joined Tim for an insightful and entertaining conversation on coding with AI that touched on everything from computer graphics in the ’90s to desire paths and why getting riled up is the key to writing a good blog post. Steve and Tim spent a lot of time discussing the wider context around Gas Town, including why Steve sees it as an enterprise tool—and as an executive assistant who takes on the mundane work so you can focus on the important problems. They also covered agent orchestration and the evolution of coding, using Steve’s eight-level framework; AI vampires and how exhausting it is to work with a stable of agents (Steve’s taking two naps a day!); why you might find yourself on the wrong side of the bitter lesson, and what to do about it; the reason Steve “wouldn’t touch [OpenClaw] with somebody else’s 10-foot pole”; why he’s adamant that developers need to overhaul their mental models from structure and framework cognition to just letting AI do its thing—and why he’s not even looking at the code anymore; and much, much more. “The big takeaway,” Steve told Tim, “is that there’s always more work. It doesn’t matter how superhumanly good your helpers get; you’re just going to want to do something bigger. Our ambition will always outstrip our compute.” Check out Tim’s takeaways from the conversation, plus clips, on Radar.

About

A series of candid conversations designed to slow down, dig deeper, and share real insights you can build on. Get the story behind the show here: https://www.oreilly.com/radar/more-slowly/

You Might Also Like