System Prompt

Peter

System Prompt is a podcast about what’s actually happening in AI. Not hype. Not surface-level takes. We break down how AI is changing software, SaaS, infrastructure, and the way systems are built focusing on real-world tradeoffs, architecture decisions, and where the value is actually shifting. If you’re building, deploying, or thinking seriously about AI, this is for you.

  1. 1d ago

    Why China is winning the AI race (Qwen3.8:27B is amazing)

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/why-china-is-winning-the-ai-race A 27 billion parameter model should not be competing with frontier AI. But Qwen3.8:27B is making that comparison a lot less ridiculous. In Episode 22 of System Prompt, Peter and Val look at what a model this size can actually do in practical use. Instead of just discussing benchmarks, Peter runs Qwen3.8:27B locally inside Pi Code and gives it a real task during the episode: build a comparative analysis workflow, create test data, work through failures, validate the results, and produce a usable report. It finishes before the episode ends. The bigger question is not whether Qwen replaces frontier models. It is how much work no longer needs a frontier model at all. WHAT WE DISCUSS • Why Qwen3.8:27B matters • Running capable AI locally • Coding and long-running tasks • Tool use and agent workflows • Using local AI for business work • Where smaller models still fall short • Executor models vs heavy reasoning models • Routing harder work to frontier AI • Dense models vs mixture-of-experts • How local AI changes cost and infrastructure KEY TAKEAWAYS 27B MODELS CAN DO REAL WORK Qwen3.8:27B is small enough to run on prosumer hardware while still being capable of coding, tool use, structured analysis, and longer-running tasks. THE HARNESS MATTERS The model does not work alone. Inside Pi Code, Qwen can inspect its environment, create tools, write code, run tests, find problems, and continue working toward a finished result. BUSINESS WORK IS A REAL USE CASE During the episode, Qwen builds a comparative analysis capability from scratch. It creates test data, cleans and normalizes information, performs the analysis, and generates graphs from the results. The output still needs human review, but the model can take meaningful execution work off someone's plate. LOCAL DOES NOT HAVE TO REPLACE FRONTIER The goal is not to eliminate Claude, ChatGPT, or other frontier models. A local model can handle well-defined execution while more ambiguous or difficult work routes to a frontier model when necessary. GOOD SPECS MATTER Qwen performs best when the task is clear. A human or stronger model can define the plan and requirements, then hand execution to the smaller model. That makes routing and task design increasingly important. THE FUTURE IS HYBRID Local models will not win every task, and frontier models are not going away. But as smaller models improve, more work can happen locally while frontier models become the escalation path instead of the default. CHAPTERS 00:00 Episode 22 01:12 Why Qwen3.8:27B? 03:35 Comparing 27B to Frontier AI 05:37 What Can You Actually Do With It? 07:50 Building a Workflow Live 14:09 What Smaller Models Mean 17:11 Local AI Economics 20:26 Internal Business Assistants 23:02 Routing to Frontier Models 26:21 Where Qwen Falls Short 28:36 Do You Need the Best Model? 38:56 Why the Future Is Hybrid 41:55 The Finished Analysis 43:44 Dense vs Mixture-of-Experts 46:36 What Local AI Can Replace 51:01 What 27B Enables Today

  2. Aug 13

    Evals: How Do You Know Which AI Model to Trust?

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/how-to-know-which-ai-model-to-trust The AI model at the top of a leaderboard may not be the best model for your system. Because the leaderboard is not testing your system. In Episode 21 of System Prompt, Peter and Val break down AI evals: what benchmarks measure, why the harness matters, and how to test models against the work you actually expect them to do. Peter walks through a custom eval across more than 20 local and open models covering tool calling, extraction, instruction following, and real-world coding tasks. The results were surprising. Smaller models matched or beat much larger ones. Turning reasoning on sometimes made performance worse. The bigger lesson: an eval measures more than the model. Quantization, runtime, token budgets, reasoning settings, parsers, and timeouts can all affect the result. WHAT WE DISCUSS • What AI evals actually measure • Why leaderboards only tell part of the story • How the harness changes model performance • Quantization, runtimes, and configuration • Building evals around real workloads • Tool calling, extraction, instruction following, and coding • Why repetition and consistency matter • Thinking vs non-thinking configurations • Routing tasks to different models • Finding problems in your own system KEY TAKEAWAYS THE BEST MODEL DEPENDS ON THE JOB A benchmark measures performance on a particular test. It does not automatically tell you which model is best for your application. A coding agent, extraction pipeline, chatbot, and tool-using agent all need different things. Start with the workload, then choose the eval. THE HARNESS IS PART OF THE RESULT Models do not operate alone. The harness creates prompts, exposes tools, manages token limits, parses responses, and decides whether a task succeeded. Change the harness, configuration, quantization, or runtime and you can change the result. TEST THE MODEL YOU ARE ACTUALLY RUNNING A full-precision benchmark is useful reference data, but it is not the same experiment as running a Q4 model through a local runtime. Your production configuration is part of the evaluation. REPETITION MATTERS One successful run does not prove reliability. Running tasks multiple times exposes models that score well once but behave inconsistently. For production systems, stability matters. REASONING IS NOT ALWAYS BETTER Thinking modes helped some models and hurt others. In some cases reasoning increased token use, hit time or output budgets, or reduced consistency. The right configuration has to be measured against the task. EVALS ENABLE ROUTING The best architecture may not use one model for everything. A smaller model may handle chat, extraction, or tool calling while another handles coding or harder reasoning. Once you know where each model succeeds and fails, routing stops being guesswork. EVALS TEST YOUR SYSTEM TOO The eval process also exposed problems in Peter's own gateway and harness. Some apparent model failures were really token limits, timeouts, parsing issues, or infrastructure problems. CHAPTERS 00:00 Episode 21 and the 1% 00:48 What Are AI Evals? 02:48 Model Capability and Benchmarks 05:07 Why the Harness Matters 09:48 Quantization and Fair Comparisons 11:30 Building a Custom Eval Suite 19:04 Repetition and Reliability 20:50 The Model Results 23:17 When Thinking Hurts Performance 29:55 Accuracy Versus Token Cost 30:55 Routing Tasks to Different Models 37:00 Evals Finding Bugs in the System 40:36 Closing Thoughts

  3. Aug 6

    Conversation about Quantization(Also about hosting DeepSeekv4:Flash)

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/how-a-284-billion-parameter-model-fits-on-one-machine A 284-billion-parameter AI model should not fit on one local machine. DeepSeek V4 Flash does. In Episode 20 of System Prompt, Peter and Val explore how quantization, mixed precision, importance-aware compression, and speculative decoding make it possible to run a massive mixture-of-experts model on hardware such as a single DGX Spark. Peter breaks down how Antirez compressed DeepSeek V4 Flash to about 81 GB while preserving enough reasoning, coding, and tool-use ability to remain useful. Most parameters sit inside routed experts using roughly two-bit quantization. More sensitive components remain at Q8, FP16, or FP32. An importance matrix helps identify which compression errors are most likely to damage the model's behavior. Peter also demonstrates the model running live through his local agent infrastructure at roughly 20 to 30 tokens per second. WHAT WE DISCUSS • Running DeepSeek V4 Flash on one machine • Quantization and model compression • Mixture-of-experts architecture • Importance matrices and mixed precision • Speculative decoding, dSpark, and Dwarf Star • Continuous local inference • Local models for coding, validation, and automation • Human oversight and self-improving systems KEY TAKEAWAYS QUANTIZATION PRESERVES USEFUL BEHAVIOR Quantization lowers the precision used to represent model weights. That reduces memory use but introduces approximation errors. The goal is not to preserve every original value. It is to preserve the behavior that makes the model useful. NOT EVERY PART SHOULD BE COMPRESSED EQUALLY The routed experts contain most of the model's parameters and receive the most aggressive compression. Sensitive components stay at higher precision because errors there can affect the model more broadly. IMPORTANCE MATRICES HELP PROTECT QUALITY An importance matrix uses real model activations to estimate which weight dimensions matter most during inference. Calibration matters because a model tuned only for conversation may become less reliable during coding, tool use, structured output, reasoning, or long-context retrieval. FITTING THE MODEL IS ONLY THE FIRST PROBLEM A model fitting into memory does not automatically make it fast, scalable, or production-ready. This implementation is mainly suited to one user with low concurrency. SPECULATIVE DECODING IMPROVES SPEED A draft mechanism proposes several future tokens. The full model verifies them, accepts the longest valid sequence, and rejects the rest. Using dSpark and Dwarf Star, Peter reports about 20 to 30 generated tokens per second on a single DGX Spark. LOCAL INFERENCE CHANGES THE ECONOMICS Local models can support research, validation, coding, monitoring, and automation without creating an API charge for every generated token. The hardware still has costs, but inference becomes owned capacity instead of a metered service. CHAPTERS 00:00 Celebrating Episode 20 01:10 Committing to 100 Episodes 01:59 Introduction to Quantization 03:31 Comparing Agents and Live Demos 04:06 DeepSeek V4 Flash 05:23 Quantization and Model Compression 09:07 Importance Matrices 10:12 Q Weights and Mixed Precision 13:36 Maximizing Local Model Output 16:59 Speculative Decoding 17:52 Live Model Demonstration 20:06 dSpark and Dwarf Star 21:39 Future of Quantization 24:37 Practical Local Model Applications 29:04 Continuous Inference and Validation 32:29 Self-Improving Models 36:00 Fear, Competition, and Market Share 37:06 Hope for Local AI

  4. Jul 29

    This Is Why Copilot Adoption Is Failing

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/why-businesses-dont-trust-ai-agents-yet AI agents do not earn trust because they work once. They earn trust by working consistently, recovering from failure, and completing the workflows employees depend on. In Episode 19 of System Prompt, Peter and Val explore building business agents with Microsoft 365 and Copilot Studio. Peter demonstrates a COO-style agent grounded in SharePoint data. It reviews operational information, identifies risks, and hands report creation to a specialized subagent. Then the report-writing agent fails. The episode becomes a real-time look at what happens when an agent that worked previously suddenly stops, returns a generic system error, and gives the user no clear recovery path. WHAT WE DISCUSS • Building agents with Microsoft 365 • Grounding agents in SharePoint data • Using subagents for specialized tasks • Limiting tool surfaces • Running evaluations and reviewing traces • Troubleshooting failed agent handoffs • Comparing Copilot Studio with Claude Cowork • Why reliability affects adoption KEY TAKEAWAYS MICROSOFT OFFERS A PRACTICAL STARTING POINT For businesses already using Microsoft 365, Copilot Studio reduces some of the work around authentication, permissions, distribution, and access to business data. The opportunity is not just another AI model. It is an agent operating inside the environment employees already use. AGENTS NEED REAL RESPONSIBILITIES The COO Coach reviews operational information, identifies risks, prepares weekly summaries, and delegates report creation. That connects the agent to a real business process instead of using it as a general chatbot. SEPARATION OF DUTIES MATTERS The main agent handles analysis. The report-writing agent creates the final document. Smaller tool surfaces make workflows easier to understand, test, and troubleshoot. AI SHOULD SUPPORT HUMAN DECISIONS The agent identifies overdue invoices, missed milestones, declining margins, and other risks. It provides information and options without making executive decisions for the user. CONSISTENCY CREATES TRUST The workflow had worked several times before the episode. Nothing meaningful changed, but it began returning a generic system error during the live demo. It eventually worked again after settings were changed, saved, changed back, and saved again. That is not a dependable recovery process. When a three-minute workflow suddenly requires 25 minutes of troubleshooting, the value disappears. Consistency creates trust. Trust creates adoption. RELIABILITY IS THE USER EXPERIENCE Employees will not depend on an agent that works unpredictably before a meeting, review, or deadline. The manual process may be slower, but users will return to it when it is more dependable. A capable agent is not enough. It also has to work when people need it. CHAPTERS 00:00 — Introduction 01:04 — Building Agents with Microsoft 365 03:40 — Creating the COO Coach 11:37 — Live Copilot Studio Demo 18:45 — Troubleshooting and Evaluations 31:59 — Copilot Studio vs. Claude Cowork 34:24 — Consistency and Reliability 40:06 — Impact on Real Workflows 47:49 — Loss of Confidence 50:14 — Why Trust Drives Adoption 59:41 — Call for Better Reliability ABOUT SYSTEM PROMPT System Prompt covers AI infrastructure, automation, agents, enterprise platforms, training, and practical implementation.

  5. Jul 15

    Your Company Bought AI. Why Isn’t Anyone Using It?

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/why-no-one-uses-your-ai Buying AI tools does not create adoption. Employees need to understand why the tools matter, how they connect to their work, and what success is supposed to look like. In Episode 18 of System Prompt, Peter and Val are joined by Jonny Havey of E-Learning Partners to discuss AI adoption, employee education, training ROI, and the role of clear KPIs. The conversation examines why training programs fail when they are disconnected from business outcomes and how organizations can measure whether AI education is actually improving performance. WHAT WE DISCUSS • Why buying AI tools does not guarantee adoption • The role of education in successful implementation • Why training must connect to business outcomes • How KPIs measure training effectiveness • Why ROI is broader than direct financial return • How AI is changing workplace education • Why employees need role-specific training • The difference between intelligence and wisdom KEY TAKEAWAYS AI ADOPTION REQUIRES MORE THAN ACCESS Giving employees an AI license does not mean they will use it effectively. They need to understand which problems it can solve, when it should be used, what data is appropriate to provide, and how outputs should be evaluated. TRAINING NEEDS CLEAR KPIS A training program should be tied to a measurable outcome. That could include faster task completion, fewer errors, reduced onboarding time, higher adoption, stronger confidence, or improved customer experience. The KPI should be defined before the training is created. ROI IS NOT ONLY FINANCIAL Training may create value through saved time, reduced risk, stronger consistency, better retention, and improved decision-making. Those outcomes may affect revenue or cost later, even when the return is not immediately financial. TRAINING SHOULD ALIGN WITH ORGANIZATIONAL GOALS Generic AI workshops often fail because they are disconnected from the work employees actually perform. Different teams have different workflows, risks, and responsibilities. Training becomes more useful when it is built around real business processes and expected outcomes. AI SHOULD SUPPORT WISDOM, NOT JUST OUTPUT AI can generate information quickly, but information alone does not create good judgment. Employees still need the experience and context required to evaluate recommendations and understand consequences. CHAPTERS 00:00 — The Importance of ROI and KPIs 05:16 — Aligning Training Programs with KPIs 08:36 — The Evolution of E-Learning Partners 24:19 — The Impact of AI Adoption on Education 45:42 — The Role of E-Learning Partners ABOUT JONNY HAVEY Jonny Havey is the co-founder of E-Learning Partners and host of the Learning Transformed podcast. https://elearningpartners.com WATCH THE EPISODE https://youtu.be/M3iRj99gy7Y ABOUT SYSTEM PROMPT System Prompt covers AI infrastructure, automation, agents, enterprise platforms, training, and practical implementation.

  6. Jul 8

    How AI is Transforming Food Discovery with Stupid Good AI

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/food-discovery-stupid-good-ai Restaurant discovery is still dominated by ratings, generic lists, and search results that often fail to reflect what a person actually wants. In Episode 17 of System Prompt, Peter and Val are joined by David Leibner, founder of Stupid Good AI, to discuss how artificial intelligence could create a more personalized way to discover food, restaurants, and local experiences. The conversation explores how Stupid Good AI combines community feedback, data science, and AI to move beyond static reviews and broad “best of” lists. David also discusses building a vertical AI product, the changing software-development process, open models, and how user behavior can improve recommendations over time. WHAT WE DISCUSS • Why restaurant discovery still feels broken • How AI can personalize food recommendations • Why ratings alone do not capture individual preferences • How community feedback improves discovery • Building a vertical AI product around a specific problem • The role of data science and open models • How AI is changing software development • Turning local restaurant data into useful recommendations KEY TAKEAWAYS FOOD DISCOVERY NEEDS MORE CONTEXT A highly rated restaurant is not automatically the right recommendation for every person. Useful discovery should consider taste, location, occasion, dietary needs, budget, atmosphere, and previous preferences. COMMUNITY DATA CAN IMPROVE RECOMMENDATIONS User feedback gives the system more than a single star rating. Over time, patterns across reviews, preferences, and behavior can help produce recommendations that are more relevant to each user. VERTICAL AI PRODUCTS CAN GO DEEPER Stupid Good AI focuses on one specific problem rather than trying to become a general-purpose assistant. That narrower scope creates an opportunity to build better data, workflows, and experiences around food discovery. AI DOES NOT REPLACE PRODUCT THINKING Models can support recommendations, classification, search, and personalization. The product still needs strong data, useful interfaces, reliable feedback loops, and a clear understanding of the user’s problem. ABOUT DAVID LEIBNER David Leibner is the founder of Stupid Good AI, a platform focused on improving how people discover restaurants and local food experiences. Learn more: https://stupidgood.ai Early-access users can mention System Prompt when contacting the Stupid Good AI team. ABOUT SYSTEM PROMPT System Prompt covers AI infrastructure, automation, agents, local models, enterprise platforms, and practical implementation.

  7. Jul 1

    Enterprise AI Training Is Failing Because Tool Training Is Not Enough

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/enterprise-ai-training Enterprise AI training is often reduced to showing employees how to use a chatbot or write better prompts. That is not enough for real implementation. In Episode 16 of System Prompt, Peter and Val examine why businesses need a deeper understanding of AI tools, ecosystems, workflows, memory, data, and operational responsibility. The conversation explores the role of a head of AI, forward-deployed engineers, and internal leaders who can connect business problems with practical systems. AI education should not be treated as a one-time workshop. Models, tools, risks, and capabilities change continuously, which means organizations need an ongoing process for learning, testing, and implementation. WHAT WE DISCUSS • Why basic prompt training is not enough • Starting with business problems instead of tools • Understanding AI ecosystems and integrations • The role of memory and context • Why employees need role-specific training • What a head of AI should own • How forward-deployed engineers support implementation • The gap between experimentation and production KEY TAKEAWAYS TOOL TRAINING IS NOT AI EDUCATION Teaching employees how to open a chatbot may create familiarity, but it does not explain how AI fits into workflows, where the risks are, what data can be used, or how outputs should be verified. START WITH THE BUSINESS PROBLEM Organizations should identify slow, expensive, repetitive, or error-prone processes before selecting an AI tool. The goal is to improve an outcome, not simply add AI. ECOSYSTEM UNDERSTANDING MATTERS AI tools interact with data, identity systems, permissions, applications, APIs, memory, retrieval, and existing workflows. Businesses need people who understand how those components connect and where failures may appear. THE HEAD OF AI IS AN OWNERSHIP ROLE A head of AI should connect business priorities, education, governance, implementation, measurement, and technical teams. Without clear ownership, adoption becomes fragmented across departments. FORWARD-DEPLOYED ENGINEERS CLOSE THE GAP These engineers work closely with users, processes, and existing infrastructure to turn business problems into working systems. Their value comes from combining technical execution with operational understanding. AI EDUCATION MUST CONTINUE Organizations need ongoing training, testing, documentation, and feedback. The goal is not to make every employee an AI expert. It is to make each employee competent within the boundaries of their role. CHAPTERS 00:00 — The Current State of Enterprise AI Education 07:27 — Identifying Problems and Solutions 19:01 — The Need for Deep Understanding 25:09 — The Role of Forward-Deployed Engineers 36:21 — The Challenge of AI Implementation WATCH THE EPISODE https://youtu.be/HdcDCQS1aRE ABOUT SYSTEM PROMPT System Prompt covers AI infrastructure, automation, agents, enterprise platforms, training, and practical implementation.

  8. Jun 25

    What Fable 5’s Removal Reveals About AI Model Dependence

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/fable-5-removal-model-dependence What happens when a model your workflow depends on is suddenly removed, restricted, or made more expensive? In Episode 15 of System Prompt, Peter and Val discuss what Fable 5’s removal reveals about dependence on a single AI provider or model. The conversation explores alternative models, open models, operating costs, and the mistaken belief that openly available models are automatically free to use. A model may be downloadable without a per-token fee, but running it still requires hardware, hosting, power, engineering, maintenance, and monitoring. The broader lesson is that businesses should evaluate AI systems based on sustainable economics, portability, and the cost of completing real work. WHAT WE DISCUSS • The impact of Fable 5’s removal • The risks of depending on one model provider • Why businesses need model alternatives • Open models compared with proprietary frontier models • Why openly available models are not free to operate • Hardware, hosting, and maintenance costs • Sustainable economics for AI products • How open models can support proprietary products KEY TAKEAWAYS MODEL ACCESS CAN CHANGE Providers can remove models, alter subscription access, introduce limits, change pricing, or replace one model with another. Production workflows need fallback options and a clear migration path. ALTERNATIVE MODELS CREATE RESILIENCE Teams should understand which tasks require frontier capability and which can be handled by smaller, local, or openly available models. Routing and evaluation make it easier to move workloads when access or economics change. OPEN MODELS ARE NOT FREE TO OPERATE Organizations still need to account for hardware, cloud hosting, energy, storage, engineering, updates, security, monitoring, and support. The correct comparison is total cost per completed task. ECONOMICS SHOULD DRIVE MODEL SELECTION The most capable model is not always the most efficient model. Businesses should compare output quality, retries, latency, human review, integration effort, and operating cost. OPEN MODELS CAN POWER PROPRIETARY PRODUCTS A company can build commercial value around an open model through its data, workflow, integrations, evaluations, user experience, and operational system. The model may be open while the complete product remains differentiated. WATCH THE EPISODE https://youtu.be/xQ3RtLr3_t4 ABOUT SYSTEM PROMPT System Prompt covers AI infrastructure, automation, agents, local models, enterprise platforms, and practical implementation.

See All (22)

About

System Prompt is a podcast about what’s actually happening in AI. Not hype. Not surface-level takes. We break down how AI is changing software, SaaS, infrastructure, and the way systems are built focusing on real-world tradeoffs, architecture decisions, and where the value is actually shifting. If you’re building, deploying, or thinking seriously about AI, this is for you.

Information

  • Creator
    Peter
  • Years Active
    2026
  • Episodes
    22
  • Rating
    Explicit
  • Copyright
    © 2026 Peter
  • Show Website
    System Prompt