MLOps.community

Demetrios
  • 5.0 (2)
  • TECHNOLOGY
  • UPDATED WEEKLY

Relaxed Conversations around getting AI into production, whatever shape that may come in (agentic, traditional ML, LLMs, Vibes, etc)

  1. Voice Agent Use Cases

    4 DAYS AGO

    Voice Agent Use Cases

    This episode is brought to you by the MLflow team. Check out more information at MLflow.org. What does it actually take to build voice AI at a billion-interaction scale? This episode features an ex-Amazon voice AI engineer who built customer support systems handling 2 billion+ interactions — now working on next-gen voice agent platforms. Anurag digs deep into the real engineering tradeoffs, design patterns, and use cases that separate production-grade voice agents from demos. Voice Agent Use Cases // MLOps Podcast #372 with Anurag Beniwal, Member of the Technical Staff at ElevenLabs 🎙️ Topics covered: 🔹 Cascaded vs. speech-to-speech — Why cascaded systems still win in production, and how to make them feel natural without sacrificing control 🔹 Latency masking — Foreground/background model architecture and how to buy yourself time while deep retrieval runs 🔹 Constellation of models — Using Haiku for tool calling, fine-tuned smaller models for response generation, and why "one model for everything" breaks at scale 🔹 Turn-taking & ASR challenges — Why voice is harder than chat: accents, noise, silence detection, and domain-specific fine-tuning 🔹 Level 1 vs Level 2 customer support — Why today's agents max out at Level 1 and what it takes to capture Level 2 expert judgment 🔹 Inbound vs. outbound sales agents — Where voice agents are already winning, and why inbound lead qualification beats cold outbound 🔹 Booking, reservations & concierge — The clearest near-term wins for voice agents across hospitality, home services, and SMBs 🔹 Continual learning from natural language feedback — How to build agents that improve from real operator feedback without ML expertise 🔹 Conversational TTS — Why passing full conversation history to your TTS model changes everything for tone consistency 🔹 User tiers for voice platforms — Non-technical business owners vs. developers vs. enterprise: why one interface doesn't fit all. If you're building production voice agents, evaluating voice AI vendors, or scaling AI-first customer support — this episode is packed with hard-won lessons from someone who's done it at Amazon scale. 🔗 Links & Resources: MLOps.community: https://mlops.communityGoogle Scholar: https://scholar.google.com/citations?user=g_QB5WgAAAAJ&hl=en&o Amazon science page: https://www.amazon.science/author/anurag-beniwal Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter MLOps GPU Guide: https://go.mlops.community/gpuguide ⏱️ Timestamps [00:00] Cascaded Systems Control Challenge [05:35] Voice vs Chat Complexity [14:16] MLflow's open source platform [15:03] AI Model Constellations [23:00] Model Constellations Use Cases [31:40] Voice vs Text Context [33:54] Voice as Thought Capture [42:11] Cascaded vs Speech-to-Speech Debate [50:02] Wrap up

    51 min
  2. The Creator of Superpowers: Why Real Agentic Engineering Beats Vibe Coding

    24 APR

    The Creator of Superpowers: Why Real Agentic Engineering Beats Vibe Coding

    Jesse Vincent is the Founder & CEO of Prime Radiant and creator of Superpowers — the most-used Claude Code plugin in the world. He built the first agentic software development methodology from scratch while managing MIT interns in the early 2000s, and hasn't written a line of code manually since October. The Creator of Superpowers: Why Real Agentic Engineering Beats Vibe Coding // MLOps Podcast #373 with Jesse Vincent, Founder & CEO of Prime Radiant In this conversation, Jesse walks Demetrios through the full Superpowers system: why he thinks most developers are still approaching agentic coding wrong, how he designs skills that force LLMs to stop rationalizing and actually follow rules, and what he's building next at Prime Radiant — including Green Field, an unreleased tool for reverse-engineering legacy codebases into specs. This one is for developers who want to go beyond "vibe coding" and build AI-assisted workflows that actually scale. 🔧 Topics Covered 🧠 The Superpowers Methodology — How the brainstorming skill extracts what you actually want before you hand work to an agent, and why most developers skip this step 📋 Spec-Driven Development & Plan Files — Why Jesse insists on TDD, DRY, and YAGNI for every agentic task, and how planning skills generate per-task context blocks agents can actually execute on 🐛 Debugging with Agents — Jesse's systematic approach to root cause analysis, reproduction cases, and the 30 years of debugging instinct he's baked into a skill 🔄 Pressure Testing LLM Skills — How Claude fires up sub-agents and stress-tests its own rules to catch rationalization before it shows up in production 🛠️ Clearance IDE — Jesse's new Markdown-native development environment built for humans working alongside AI, with a history pane for file navigation 📦 Green Field (Unreleased) — A toolset for turning old codebases or built products into clean specs — not yet public but dropping soon from Prime Radiant 🧑‍💼 Management as the Magic Trick — Why the real unlock of tools like Superpowers is that they make every developer a manager, and why that transition is hard the first time ⚖️ Software Ethics in the Agent Era — Reverse engineering, license washing, open source cloning, and whether the value of software itself is collapsing 🔗 Links & Resources Prime Radiant: [https://prime-radiant.com](https://prime-radiant.com/) Superpowers on GitHub: https://github.com/prime-radiant-inc Clearance IDE: https://github.com/prime-radiant-inc (check repo) MLOps.community Slack: https://go.mlops.community/slack MLOps.community website: [https://mlops.community](https://mlops.community/) ⏱️ Timestamps [00:00] Greenfield Toolset Insights [00:27] Superpowers Kit Evangelism [08:06] Hyperbolic's GPU Cloud [17:48] Debugging Skill Creation [22:12] Skill Extraction Strategy [31:15] Smallest Harness [41:06] Software supply chains [48:56] Visual Precision Challenges [54:09] Creative Feedback Loops [1:04:24] MLflow's Gen AI [1:05:55] Wrap-up

    1hr 7min
  3. It's 2026, and We're Still Talking Evals

    21 APR

    It's 2026, and We're Still Talking Evals

    Maggie Konstanty is an AI Product Manager at Prosus, one of the world's largest consumer internet companies, where she builds and evaluates AI agents for food ordering and ecommerce at scale. She's been inside the messy reality of LLM evaluation longer than most — and her take is unfiltered. It's 2026, and We're Still Talking Evals // MLOps Podcast #372 with Maggie Konstanty, AI Product Manager at Prosus 🧪 Why accuracy metrics lie — Maggie breaks down why "95% accurate" tells you almost nothing about whether your agent is actually working in the real world, and what to measure instead. 🏗️ Pre-ship vs. production evals — Your eval suite before launch will not survive first contact with real users. Maggie explains the structural disconnect and how to close the gap. 👻 The silent failure: user drop-off — Users who are unhappy don't complain — they just leave. Discover why drop-off analytics are one of the most underutilized eval signals in production. 🎯 Instruction to fail: the 20-evaluator trap — Setting up 20 types of evaluators not connected to your product goal is a fast path to wasted time. How to design evals that are tied to real outcomes. 🍽️ The "surprise me" edge case — A real example from Prosus's food ordering agent and what it reveals about how users actually behave vs. how PMs imagine they do. 🤖 LLM-as-a-judge: the limits — Why Maggie doesn't lean on LLM-as-a-judge for accuracy measurement, and what approaches she uses instead for production-grade evaluation. 🛠️ Arize/Phoenix & eval tooling critique — A candid take on the current state of eval platforms, why she spent a whole day fighting the UI, and why mature teams often go back to custom code. 🧬 Eval as team DNA — Evals aren't a launch checklist. Maggie makes the case that they need to be a constant practice embedded in team culture — and why alignment on "what good looks like" is harder than any technical implementation. 🔢 When to stop optimizing — What happens when your eval score approaches 100%, and how to know when it's time to shift focus to a different metric or flow. 💬 Red teaming with incentives — A fun tactic: running adversarial eval sessions where engineers compete to break your agent for an Amazon gift card. This is required watching for AI PMs, ML engineers, and applied AI teams who have moved past "getting evals set up" and are now struggling with making them actually matter.--- 🔗 Links & Resources Maggie Konstanty on LinkedIn: https://www.linkedin.com/in/maggie-konstanty Prosus: [https://www.prosus.com](https://www.prosus.com/) MLOps.community: [https://mlops.community](https://mlops.community/) Arize AI / Phoenix (mentioned): [https://arize.com](https://arize.com/) / [https://phoenix.arize.com](https://phoenix.arize.com/) MLOps.community Slack: https://go.mlops.community/slack ⏱️ Timestamps [00:00] Evaluations and User Alignment [00:18] Eval Lifecycle in Production [06:05] LLM Accuracy and Judging [15:30] Evals vs Tests in AI [22:39] Profanity as Frustration Signal [29:23] Impact-weighted performance [32:22] Eval Tooling Pros and Cons [38:10] Build vs Buy Dilemma [39:35] Wrap up

    41 min
  4. Why Agents are Driving Software Development to the Cloud

    17 APR

    Why Agents are Driving Software Development to the Cloud

    This episode is brought to you by Hyperbolic and the MLflow team. Check out more information at hyperbolic.ai and MLflow.org. Why AI Coding Agents Are Moving to the Cloud — With Zach Lloyd, CEO of Warp Zach Lloyd is the founder and CEO of Warp, the AI-native terminal and agentic development platform trusted by over a million developers. Before Warp, Zach was a product lead at Google on Google Docs — giving him a uniquely deep intuition for what it means to build truly collaborative developer tools at scale. Why Agents are Driving Software Development to the Cloud // MLOps Podcast #371 with Zach Lloyd, CEO of Warp What we cover: 🏗️ Why agents belong in the cloud, not local sandboxes — Zach breaks down why the "set up a local dev box for your agent" approach is fundamentally flawed and what cloud-native agent execution actually looks like in practice. 🚀 GitHub is losing collaborative code review — One of the episode's sharpest takes: the hero features of GitHub, like collaborative code review, are migrating into agent workbenches. Zach explains why this shift is structural, not cyclical. 📱 "Just-in-time apps" are replacing SaaS — The era of long-lived, learn-to-use-it software may be ending. Zach argues that agents will generate ephemeral, purpose-built interfaces on demand — and why most current app categories are at risk. 🤖 Introducing Oz — Warp's cloud orchestration platform — A first look at how Oz works, how Demetrios is already using it to automate podcast production, and what multi-agent orchestration looks like in a real team environment. 👁️ Agent observability and why it matters — Debugging, compliance, context management, and handoff/steering: Zach outlines the three pillars every engineering team needs before trusting agents with production work. 🔐 Agent chaos is real — access control for AI — Why giving agents too much context is just as dangerous as giving them too little, and how Warp thinks about scoped agent permissions as you scale. 📦 SaaS for agents will look nothing like SaaS for humans — The 25-year investment in human-friendly UI is irrelevant for agents. Zach explains what the new infrastructure layer for AI workers will actually need. ⚡ Open-weight models will commoditize the coding agent space — With Nvidia investing $2B in open-weight models, Zach believes the current cost advantage that frontier labs hold is temporary — and how Warp is positioning for that world. 🧩 Multi-agent orchestration patterns — Parallel agents, agent-to-agent handoffs, and why there's no single "right" pattern yet. Warp's Oz platform is being built for flexibility, not prescription. This episode is essential for engineering leaders, platform engineers, and any developer trying to understand where their daily workflow is headed in the next 18 months. 🔗 Links & Resources: Warp: https://www.warp.dev Warp Oz platform: https://oz.dev Zach Lloyd on X/Twitter: https://x.com/zachlloyd MLOps Community: https://mlops.community MLOps Community Slack: https://go.mlops.community/slack ⏱️ Timestamps [00:00] Agentic Coding Review Shift [00:29] Warp Collaboration vs Sandboxes [05:22] Continuous Co-Creation in Teams [07:00] Hyperbolics GPU Cloud [07:56] Skill Governance Framework [14:41] Agents vs Browsers Analogy [21:31] PR Provenance in Warp [27:58] Agent System Commandments [37:44] Harness vs ADE [42:03] Adversarial Review Technique [45:26] GitHub Limitations for Agents [49:07] MLflow's GenAI [50:06] Wrap up

    51 min
  5. The Modern Software Engineer

    14 APR

    The Modern Software Engineer

    This episode is brought to you by the MLflow team. Check out more information at MLflow.org. Mihail Eric is Head of AI at Monaco and Adjunct Lecturer at Stanford University, where he teaches CS146S: "The Modern Software Developer" — the first course in the world dedicated to how AI is transforming every stage of the software development lifecycle. With 12+ years building production AI systems at Amazon Alexa, Storia AI (YC S24), and early-stage startups, Mihail has one of the most grounded, practitioner-level takes on what it actually means to be a software engineer in 2026. The Modern Software Engineer // MLOps Podcast #370 with Mihail Eric, Head of AI at Monaco 🧠 What the modern software engineer actually looks like — why the job description has fundamentally shifted from writing code to designing systems and directing agents ⚙️ Agents require more thinking, not less — why the engineers getting the most out of coding agents are the ones who invest the most upfront in architecture, planning, and codebase structure 🎓 Inside Stanford's "Modern Software Developer" course — what Mihail teaches in the first CS course in the world focused entirely on AI-transformed software development 🏗️ From writing code to designing systems — how the best developers are repositioning themselves as architects of agentic workflows rather than line-by-line coders 🔁 The Build System: how to run agents at scale — practical lessons from building multi-agent pipelines, parallel subagent batches, and automated retrospectives📉 What junior engineers should actually focus on — the skills that remain irreplaceable and the paths that still produce strong software engineers in an AI-first world 🚀 Building Monaco's AI-native revenue engine — what it's like building AI infrastructure for a fast-moving $35M-funded startup disrupting enterprise CRM 🎯 How to ace AI engineering interviews — Mihail's framework for demonstrating real AI engineering competence beyond prompt engineering basics. Essential watching for software engineers, ML practitioners, and engineering managers who want an honest, practitioner-level view of where the profession is going — from someone who's both teaching it at Stanford and building it in production. 🔗 Links & Resources Mihail Eric on LinkedIn: https://www.linkedin.com/in/mihaileric/ Mihail's website: https://www.mihaileric.com Stanford course "The Modern Software Developer": https://themodernsoftware.dev/ Maven course — AI Software Development: From First Prompt to Production Code: https://maven.com/the-modern-software-developer/ai-course Free AI Engineer interview prep course: https://course.aiengineermastery.com/ Monaco (AI-native revenue engine): https://monaco.com MLOps.community Slack: https://go.mlops.community/slack ⏱️ Timestamps 00:00 Intro — Mihail Eric & Monaco 04:00 What has actually changed for software engineers in 2026 09:00 Inside Stanford's "Modern Software Developer" course 15:00 Why agents require more human thinking, not less 21:00 From writing code to designing systems — the architect mindset 27:00 The Build System: running agents at scale in production 33:00 What junior engineers should focus on right now 39:00 Building AI infrastructure at Monaco 44:00 How to demonstrate real AI engineering competence 49:00 Skills that will remain irreplaceable 52:00 Rapid fire/closing thoughts

    54 min
  6. We Cut LLM Latency by 70% in Production

    10 APR

    We Cut LLM Latency by 70% in Production

    Maher Hanafi is an engineering leader who went from zero AI experience to self-hosting LLMs at enterprise scale — managing GPU costs, optimizing inference with TensorRT LLM, and building an AI platform for HR tech. In this conversation, he breaks down exactly how his team cut latency by 70%, reduced GPU spend through counterintuitive scaling strategies, and navigated the messy reality of taking AI from proof-of-concept to production. How We Cut LLM Latency 70% With TensorRT in Production // MLOps Podcast #369 with Maher Hanafi, SVP of Engineering at Betterworks Key topics covered: The AI Iceberg — Why the invisible work behind AI (performance, latency, throughput, cost, accuracy) is harder than building the features themselves GPU Cost Optimization — How upgrading to more expensive GPUs actually saved money by reducing total runtime hours TensorRT LLM Deep Dive — Rewiring neural networks to match GPU architecture for 50-70% latency reduction Cold Start Solutions — Using AWS FSx, baking models into container images, and cutting minutes off spin-up times KV Cache & In-Flight Batching — Why using one model per GPU with maximum KV cache beats cramming multiple models together Scheduled & Dynamic Scaling — Pattern-based scaling for HR tech workloads (nights, weekends, end-of-quarter spikes) Verticalized AI Platform — Building horizontal AI infrastructure that serves multiple HR product verticals AI Engineering Lab — How junior vs. senior engineers adopted AI coding tools differently, and the cultural shift that followed Agentic Coding in Practice — Navigating AI coding agent costs, quality control, and redefining the SDLC Chinese Models & Compliance — Why enterprise customers block DeepSeek/Qwen and the geopolitics of model training data This episode is for engineering leaders building AI in production, MLOps engineers optimizing GPU infrastructure, and anyone navigating the gap between AI demos and enterprise-scale deployment. Links & Resources: TensorRT LLM: https://github.com/NVIDIA/TensorRT-LLM NVIDIA Run: ai Model Streamer (cold start optimization): https://developer.nvidia.com/blog/reducing-cold-start-latency-for-llm-inference-with-nvidia-runai-model-streamer/ vLLM vs TensorRT-LLM comparison: https://northflank.com/blog/vllm-vs-tensorrt-llm-and-how-to-run-them Timestamps: [00:00] Optimizing GPU Usage and Latency [00:21] Learning AI as Leadership [04:34] AI Cost Centers [13:56] Throughput and Infrastructure Efficiency [18:10] Scaling and Unit Economics [24:14] Championing AI ROI [36:11] Queue to Value Engine [41:30] Failed Product Features [46:12] Agentic Engineering Costs [58:49] AI Self-Hosting in Engineering [1:04:40] Wrap up

    1hr 5min
  7. Getting Humans Out of the Way: How to Work with Teams of Agents

    7 APR

    Getting Humans Out of the Way: How to Work with Teams of Agents

    Rob Ennals is the creator of Broomy, an open-source IDE designed for working effectively with many agents in parallel. He previously worked at Meta, Quora, Google Search, and Intel Research. He has a PhD in Computer Science from the University of Cambridge. Getting Humans Out of the Way: How to Work with Teams of Agents // MLOps Podcast #368 with Rob Ennals, the Creator of Broomy Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter MLOps GPU Guide: https://go.mlops.community/gpuguide // Abstract Most people cripple coding agents by micromanaging them—reviewing every step and becoming the bottleneck. The shift isn’t to better supervise agents, but to design systems where they work well on their own: parallelized, self-validating, and guided by strong processes. Done right, you don’t lose control—you gain leverage. Like paving roads for cars, the real unlock is reshaping the environment so AI can move fast. // Bio Rob Ennals is the creator of Broomy, an open-source IDE designed for working effectively with many agents in parallel. He previously worked at Meta, Quora, Google Search, and Intel Research. He has a PhD in Computer Science from the University of Cambridge. // Related Links Website: https://robennals.org/ https://broomy.org/ https://learnai.robennals.org/ (not yet announced, but should be by the time of the podcast) ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Rob on LinkedIn: /robennals/ Timestamps: [00:00] Agent Optimization Strategies [00:21] Visual Regression Explanation [05:35] Automated QA for Videos [13:05] Verification System Design [19:48] Agent Selection Strategies [30:48] Parallel Agent Management [35:30] Containerization and Cost Estimation [42:48] Shifting to Agent Orchestration [50:10] Wrap up

    51 min
  8. Fixing GPU Starvation in Large-Scale Distributed Training

    3 APR

    Fixing GPU Starvation in Large-Scale Distributed Training

    Kashish Mittal is a Staff Software Engineer at Uber, working on large-scale distributed systems and core backend infrastructure. Fixing GPU Starvation in Large-Scale Distributed Training // MLOps Podcast #367 with Kashish Mittal, Staff Software Engineer at Uber Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter MLOps GPU Guide: https://go.mlops.community/gpuguide // Abstract Kashish zooms out to discuss a universal industry pattern: how infrastructure—specifically data loading—is almost always the hidden constraint for ML scaling. The conversation dives deep into a recent architectural war story. Kashish walks through the full-stack profiling and detective work required to solve a massive GPU starvation bottleneck. By redesigning the Petastorm caching layer to bypass CPU transformation walls and uncovering hidden distributed race conditions, his team boosted GPU utilization to 60%+ and cut training time by 80%. Kashish also shares his philosophy on the fundamental trade-offs between latency and efficiency in GPU serving. // Bio Kashish Mittal is a Staff Software Engineer at Uber, where he architects the hyperscale machine learning infrastructure that powers Uber’s core mobility and delivery marketplaces. Prior to Uber, Kashish spent nearly a decade at Google building highly scalable, low-latency distributed ML systems for flagship products, including YouTube Ads and Core Search Ranking. His engineering expertise lies at the intersection of distributed systems and AI—specifically focusing on large-scale data processing, eliminating critical I/O bottlenecks, and maximizing GPU efficiency for petabyte-scale training pipelines. When he isn't hunting down distributed race conditions, he is a passionate advocate for open-source architecture and building reproducible, high-throughput ML systems. // Related Links Website: https://www.uber.com/ Getting Humans Out of the Way: How to Work with Teams of Agents // MLOps Podcast #368 with Rob Ennals, the Creator of Broomy: https://www.youtube.com/watch?v=ie1M8p-SVfM ~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~ Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore Join our Slack community [https://go.mlops.community/slack] Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register] MLOps Swag/Merch: [https://shop.mlops.community/] Connect with Demetrios on LinkedIn: /dpbrinkm Connect with Kashish on LinkedIn: /kashishmittal/ Timestamps: [00:00] Local dataset caching [00:30] Engineers Evolving Roles [04:44] GPU Resource Management [10:21] GPU Utilization Issues [21:49] More GPU War Stories [32:12] Model Serving Issues [39:58] Reflective Learning in Coding [43:23] Workflow and Reflective Skills [52:30] Wrap up

    53 min
See All (515)

Ratings & Reviews

5
out of 5
2 Ratings

About

Relaxed Conversations around getting AI into production, whatever shape that may come in (agentic, traditional ML, LLMs, Vibes, etc)

Information

  • Creator
    Demetrios
  • Years Active
    2020 - 2026
  • Episodes
    515
  • Rating
    Explicit
  • Copyright
    © Demetrios
  • Show Website
    MLOps.community

You Might Also Like