System Prompt

Peter
  • 0.0 (0)
  • Business

System Prompt is a podcast about what’s actually happening in AI. Not hype. Not surface-level takes. We break down how AI is changing software, SaaS, infrastructure, and the way systems are built focusing on real-world tradeoffs, architecture decisions, and where the value is actually shifting. If you’re building, deploying, or thinking seriously about AI, this is for you.

  1. 4d ago

    NVIDIA, Asana, and New Models

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/nvidia-asana-and-new-models NVIDIA agreeing to acquire Hugging Face changes more than who owns the biggest model repository in open-source AI. It may also make the open ecosystem harder to push around. In Episode 24 of System Prompt, Peter and Val break down NVIDIA’s $12.9 billion Hugging Face deal, Asana’s claim that Codex compressed years of engineering work into weeks, and a wave of open models getting harder to dismiss. The conversation moves past the headlines into the systems underneath them: why NVIDIA benefits from open models thriving, why AI productivity claims need methodology and labor context, and why better local models are changing how developers think about cost, privacy, and infrastructure. Peter also walks through a one-shot AI Signal dashboard built with GLM 5.3 Flash and explains why improving open models are forcing him to rethink how much scaffolding smaller models need. WHAT WE DISCUSS • NVIDIA’s $12.9 billion Hugging Face acquisition • Why NVIDIA may become a powerful defender of open-source AI • Whether Hugging Face can stay neutral across CUDA, AMD, MLX, and other runtimes • Why open models may become harder to marginalize • AI-assisted hacking, agency, and tool access • Anthropic’s pricing problem as open models improve • Why model quality alone may not protect a frontier-model moat • Asana compressing years of engineering work into weeks with Codex • Why AI productivity claims need labor and methodology context • GLM 5.3 Flash and the rise of stronger open-weight models • Building an AI news dashboard from one ambiguous prompt • Qwen, GLM, and model routing for planning, execution, and review • Why assumptions about smaller models may already be outdated KEY TAKEAWAYS NVIDIA CHANGES THE OPEN-SOURCE POWER BALANCE Lobbying against a smaller open-model company is one thing. Doing it when NVIDIA has billions invested in the ecosystem is another. If open models grow, NVIDIA is positioned to benefit. HEADLINES NEED SYSTEM CONTEXT Compressing five years of work into two weeks sounds incredible, but the model is only part of the system. Domain expertise, testing, review, infrastructure, labor, and methodology shape the result. MODEL MOATS ARE GETTING THINNER Anthropic still produces excellent models, but quality is no longer the only variable. Cost, usage limits, privacy, local deployment, and capable open models all affect where workloads go. OPEN MODELS NEED LESS HAND-HOLDING Smaller models once needed heavy scaffolding to produce reliable results. That assumption is eroding as newer models improve at reasoning, coding, review, and ambiguous product decisions. ARCHITECTURE HAS TO FOLLOW THE MODELS Systems built around yesterday’s models may not make sense for tomorrow’s. Better models change where guardrails belong, which tasks need frontier models, and how much infrastructure is necessary. CHAPTERS 00:00 NVIDIA Buys Hugging Face 05:47 Does NVIDIA Protect Open Source? 11:34 Open Models, Hacking, and Agency 18:17 Anthropic’s Shrinking Moat 22:04 The Asana Codex Story 27:16 What AI Productivity Headlines Leave Out 31:13 GLM 5.3 Flash Arrives 34:00 Building AI Signal with an Open Model 40:50 One Prompt, Better Product Decisions 43:42 Price, Privacy, and Frontier Models 49:44 Rethinking Smaller Open Models

  2. Aug 27

    AI, Accountants, and the Future of the Profession

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/ai-accountants-and-the-future-of-the-profession AI is not just changing the tools accountants use. It is changing the work itself. In Episode 23 of System Prompt, Peter and Val are joined by Peter McCarroll, founder of Fuel Accountants and creator of The AI Accountant, to talk about what AI means for accounting firms, their teams, and the future of the profession. The conversation moves past prompt training and AI hype into the harder questions: which work should be automated, what still requires human judgment, how firms should handle sensitive client data, and what happens when AI starts taking over work that used to require years of experience. Peter McCarroll also shares a real AI failure that cost his firm roughly 10 hours of rework, why accountants should train on the work instead of the tools, and why he believes firms may have only a couple of busy seasons left before the profession looks very different. WHAT WE DISCUSS • Why AI training often fails to change how people work • Training on workflows instead of individual AI tools • Why the billable hour may become a weaker measure of productivity • Where automation should stop in professional services • Human accountability for AI-generated work • What happens when AI gets accounting work wrong • Using deterministic tools like Python for calculations • Choosing between Claude, ChatGPT, Gemini, and open models • Token cost and model selection for agentic workflows • Protecting sensitive client and financial data • Shadow AI, retention, and business risk • Whether AI will reduce accounting jobs • Why professional judgment may not be the moat accountants think it is • Context engineering for client-specific financial analysis • How accounting moves from the rearview mirror to the dashboard KEY TAKEAWAYS TRAIN ON THE WORK, NOT THE TOOL Teaching someone how to use ChatGPT or Claude does not automatically change a workflow. Start with the work, redesign the process, then teach the team how AI fits into it. ACCOUNTABILITY DOES NOT GET AUTOMATED AI can perform more of the work, but the accountant still has to stand behind what reaches the client. Professional services cannot outsource responsibility to a model. AI NEEDS CHECKPOINTS A model can appear to follow instructions while quietly dropping part of the work. Verification, review, and deterministic steps matter when the output affects financial decisions. PROFESSIONAL JUDGMENT IS CHANGING Much of what experts call judgment comes from years of internalized rules and pattern recognition. AI is increasingly capable of applying those rules, which pushes human value toward context, relationships, verification, and advice. ACCOUNTING MOVES FORWARD The profession has historically reported what already happened. AI makes it possible for accountants to move closer to real-time financial insight and become a co-pilot for what the business should do next. CHAPTERS 00:00 Meet Peter McCarroll 02:41 Train on the Work, Not the Tools 04:52 AI Workflows at Fuel Accountants 07:31 What Should Never Be Automated 10:45 When AI Fails in Accounting 13:02 Making AI More Deterministic 14:28 Choosing Models and Managing Cost 19:02 Client Data, Privacy, and AI 23:46 Accountability for AI-Generated Work 26:51 Is AI Taking Accounting Jobs? 30:10 Filtering AI Noise for Accountants 32:19 Professional Judgment Is Changing 38:45 Hallucinations and Context Engineering 42:31 What Accounting Firms Should Do Now 45:42 Accounting Five Years From Now

  3. Aug 19

    Why China is winning the AI race (Qwen3.8:27B is amazing)

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/why-china-is-winning-the-ai-race A 27 billion parameter model should not be competing with frontier AI. But Qwen3.8:27B is making that comparison a lot less ridiculous. In Episode 22 of System Prompt, Peter and Val look at what a model this size can actually do in practical use. Instead of just discussing benchmarks, Peter runs Qwen3.8:27B locally inside Pi Code and gives it a real task during the episode: build a comparative analysis workflow, create test data, work through failures, validate the results, and produce a usable report. It finishes before the episode ends. The bigger question is not whether Qwen replaces frontier models. It is how much work no longer needs a frontier model at all. WHAT WE DISCUSS • Why Qwen3.8:27B matters • Running capable AI locally • Coding and long-running tasks • Tool use and agent workflows • Using local AI for business work • Where smaller models still fall short • Executor models vs heavy reasoning models • Routing harder work to frontier AI • Dense models vs mixture-of-experts • How local AI changes cost and infrastructure KEY TAKEAWAYS 27B MODELS CAN DO REAL WORK Qwen3.8:27B is small enough to run on prosumer hardware while still being capable of coding, tool use, structured analysis, and longer-running tasks. THE HARNESS MATTERS The model does not work alone. Inside Pi Code, Qwen can inspect its environment, create tools, write code, run tests, find problems, and continue working toward a finished result. BUSINESS WORK IS A REAL USE CASE During the episode, Qwen builds a comparative analysis capability from scratch. It creates test data, cleans and normalizes information, performs the analysis, and generates graphs from the results. The output still needs human review, but the model can take meaningful execution work off someone's plate. LOCAL DOES NOT HAVE TO REPLACE FRONTIER The goal is not to eliminate Claude, ChatGPT, or other frontier models. A local model can handle well-defined execution while more ambiguous or difficult work routes to a frontier model when necessary. GOOD SPECS MATTER Qwen performs best when the task is clear. A human or stronger model can define the plan and requirements, then hand execution to the smaller model. That makes routing and task design increasingly important. THE FUTURE IS HYBRID Local models will not win every task, and frontier models are not going away. But as smaller models improve, more work can happen locally while frontier models become the escalation path instead of the default. CHAPTERS 00:00 Episode 22 01:12 Why Qwen3.8:27B? 03:35 Comparing 27B to Frontier AI 05:37 What Can You Actually Do With It? 07:50 Building a Workflow Live 14:09 What Smaller Models Mean 17:11 Local AI Economics 20:26 Internal Business Assistants 23:02 Routing to Frontier Models 26:21 Where Qwen Falls Short 28:36 Do You Need the Best Model? 38:56 Why the Future Is Hybrid 41:55 The Finished Analysis 43:44 Dense vs Mixture-of-Experts 46:36 What Local AI Can Replace 51:01 What 27B Enables Today

  4. Aug 13

    Evals: How Do You Know Which AI Model to Trust?

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/how-to-know-which-ai-model-to-trust The AI model at the top of a leaderboard may not be the best model for your system. Because the leaderboard is not testing your system. In Episode 21 of System Prompt, Peter and Val break down AI evals: what benchmarks measure, why the harness matters, and how to test models against the work you actually expect them to do. Peter walks through a custom eval across more than 20 local and open models covering tool calling, extraction, instruction following, and real-world coding tasks. The results were surprising. Smaller models matched or beat much larger ones. Turning reasoning on sometimes made performance worse. The bigger lesson: an eval measures more than the model. Quantization, runtime, token budgets, reasoning settings, parsers, and timeouts can all affect the result. WHAT WE DISCUSS • What AI evals actually measure • Why leaderboards only tell part of the story • How the harness changes model performance • Quantization, runtimes, and configuration • Building evals around real workloads • Tool calling, extraction, instruction following, and coding • Why repetition and consistency matter • Thinking vs non-thinking configurations • Routing tasks to different models • Finding problems in your own system KEY TAKEAWAYS THE BEST MODEL DEPENDS ON THE JOB A benchmark measures performance on a particular test. It does not automatically tell you which model is best for your application. A coding agent, extraction pipeline, chatbot, and tool-using agent all need different things. Start with the workload, then choose the eval. THE HARNESS IS PART OF THE RESULT Models do not operate alone. The harness creates prompts, exposes tools, manages token limits, parses responses, and decides whether a task succeeded. Change the harness, configuration, quantization, or runtime and you can change the result. TEST THE MODEL YOU ARE ACTUALLY RUNNING A full-precision benchmark is useful reference data, but it is not the same experiment as running a Q4 model through a local runtime. Your production configuration is part of the evaluation. REPETITION MATTERS One successful run does not prove reliability. Running tasks multiple times exposes models that score well once but behave inconsistently. For production systems, stability matters. REASONING IS NOT ALWAYS BETTER Thinking modes helped some models and hurt others. In some cases reasoning increased token use, hit time or output budgets, or reduced consistency. The right configuration has to be measured against the task. EVALS ENABLE ROUTING The best architecture may not use one model for everything. A smaller model may handle chat, extraction, or tool calling while another handles coding or harder reasoning. Once you know where each model succeeds and fails, routing stops being guesswork. EVALS TEST YOUR SYSTEM TOO The eval process also exposed problems in Peter's own gateway and harness. Some apparent model failures were really token limits, timeouts, parsing issues, or infrastructure problems. CHAPTERS 00:00 Episode 21 and the 1% 00:48 What Are AI Evals? 02:48 Model Capability and Benchmarks 05:07 Why the Harness Matters 09:48 Quantization and Fair Comparisons 11:30 Building a Custom Eval Suite 19:04 Repetition and Reliability 20:50 The Model Results 23:17 When Thinking Hurts Performance 29:55 Accuracy Versus Token Cost 30:55 Routing Tasks to Different Models 37:00 Evals Finding Bugs in the System 40:36 Closing Thoughts

  5. Aug 6

    Conversation about Quantization(Also about hosting DeepSeekv4:Flash)

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/how-a-284-billion-parameter-model-fits-on-one-machine A 284-billion-parameter AI model should not fit on one local machine. DeepSeek V4 Flash does. In Episode 20 of System Prompt, Peter and Val explore how quantization, mixed precision, importance-aware compression, and speculative decoding make it possible to run a massive mixture-of-experts model on hardware such as a single DGX Spark. Peter breaks down how Antirez compressed DeepSeek V4 Flash to about 81 GB while preserving enough reasoning, coding, and tool-use ability to remain useful. Most parameters sit inside routed experts using roughly two-bit quantization. More sensitive components remain at Q8, FP16, or FP32. An importance matrix helps identify which compression errors are most likely to damage the model's behavior. Peter also demonstrates the model running live through his local agent infrastructure at roughly 20 to 30 tokens per second. WHAT WE DISCUSS • Running DeepSeek V4 Flash on one machine • Quantization and model compression • Mixture-of-experts architecture • Importance matrices and mixed precision • Speculative decoding, dSpark, and Dwarf Star • Continuous local inference • Local models for coding, validation, and automation • Human oversight and self-improving systems KEY TAKEAWAYS QUANTIZATION PRESERVES USEFUL BEHAVIOR Quantization lowers the precision used to represent model weights. That reduces memory use but introduces approximation errors. The goal is not to preserve every original value. It is to preserve the behavior that makes the model useful. NOT EVERY PART SHOULD BE COMPRESSED EQUALLY The routed experts contain most of the model's parameters and receive the most aggressive compression. Sensitive components stay at higher precision because errors there can affect the model more broadly. IMPORTANCE MATRICES HELP PROTECT QUALITY An importance matrix uses real model activations to estimate which weight dimensions matter most during inference. Calibration matters because a model tuned only for conversation may become less reliable during coding, tool use, structured output, reasoning, or long-context retrieval. FITTING THE MODEL IS ONLY THE FIRST PROBLEM A model fitting into memory does not automatically make it fast, scalable, or production-ready. This implementation is mainly suited to one user with low concurrency. SPECULATIVE DECODING IMPROVES SPEED A draft mechanism proposes several future tokens. The full model verifies them, accepts the longest valid sequence, and rejects the rest. Using dSpark and Dwarf Star, Peter reports about 20 to 30 generated tokens per second on a single DGX Spark. LOCAL INFERENCE CHANGES THE ECONOMICS Local models can support research, validation, coding, monitoring, and automation without creating an API charge for every generated token. The hardware still has costs, but inference becomes owned capacity instead of a metered service. CHAPTERS 00:00 Celebrating Episode 20 01:10 Committing to 100 Episodes 01:59 Introduction to Quantization 03:31 Comparing Agents and Live Demos 04:06 DeepSeek V4 Flash 05:23 Quantization and Model Compression 09:07 Importance Matrices 10:12 Q Weights and Mixed Precision 13:36 Maximizing Local Model Output 16:59 Speculative Decoding 17:52 Live Model Demonstration 20:06 dSpark and Dwarf Star 21:39 Future of Quantization 24:37 Practical Local Model Applications 29:04 Continuous Inference and Validation 32:29 Self-Improving Models 36:00 Fear, Competition, and Market Share 37:06 Hope for Local AI

  6. Jul 29

    This Is Why Copilot Adoption Is Failing

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/why-businesses-dont-trust-ai-agents-yet AI agents do not earn trust because they work once. They earn trust by working consistently, recovering from failure, and completing the workflows employees depend on. In Episode 19 of System Prompt, Peter and Val explore building business agents with Microsoft 365 and Copilot Studio. Peter demonstrates a COO-style agent grounded in SharePoint data. It reviews operational information, identifies risks, and hands report creation to a specialized subagent. Then the report-writing agent fails. The episode becomes a real-time look at what happens when an agent that worked previously suddenly stops, returns a generic system error, and gives the user no clear recovery path. WHAT WE DISCUSS • Building agents with Microsoft 365 • Grounding agents in SharePoint data • Using subagents for specialized tasks • Limiting tool surfaces • Running evaluations and reviewing traces • Troubleshooting failed agent handoffs • Comparing Copilot Studio with Claude Cowork • Why reliability affects adoption KEY TAKEAWAYS MICROSOFT OFFERS A PRACTICAL STARTING POINT For businesses already using Microsoft 365, Copilot Studio reduces some of the work around authentication, permissions, distribution, and access to business data. The opportunity is not just another AI model. It is an agent operating inside the environment employees already use. AGENTS NEED REAL RESPONSIBILITIES The COO Coach reviews operational information, identifies risks, prepares weekly summaries, and delegates report creation. That connects the agent to a real business process instead of using it as a general chatbot. SEPARATION OF DUTIES MATTERS The main agent handles analysis. The report-writing agent creates the final document. Smaller tool surfaces make workflows easier to understand, test, and troubleshoot. AI SHOULD SUPPORT HUMAN DECISIONS The agent identifies overdue invoices, missed milestones, declining margins, and other risks. It provides information and options without making executive decisions for the user. CONSISTENCY CREATES TRUST The workflow had worked several times before the episode. Nothing meaningful changed, but it began returning a generic system error during the live demo. It eventually worked again after settings were changed, saved, changed back, and saved again. That is not a dependable recovery process. When a three-minute workflow suddenly requires 25 minutes of troubleshooting, the value disappears. Consistency creates trust. Trust creates adoption. RELIABILITY IS THE USER EXPERIENCE Employees will not depend on an agent that works unpredictably before a meeting, review, or deadline. The manual process may be slower, but users will return to it when it is more dependable. A capable agent is not enough. It also has to work when people need it. CHAPTERS 00:00 — Introduction 01:04 — Building Agents with Microsoft 365 03:40 — Creating the COO Coach 11:37 — Live Copilot Studio Demo 18:45 — Troubleshooting and Evaluations 31:59 — Copilot Studio vs. Claude Cowork 34:24 — Consistency and Reliability 40:06 — Impact on Real Workflows 47:49 — Loss of Confidence 50:14 — Why Trust Drives Adoption 59:41 — Call for Better Reliability ABOUT SYSTEM PROMPT System Prompt covers AI infrastructure, automation, agents, enterprise platforms, training, and practical implementation.

  7. Jul 15

    Your Company Bought AI. Why Isn’t Anyone Using It?

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/why-no-one-uses-your-ai Buying AI tools does not create adoption. Employees need to understand why the tools matter, how they connect to their work, and what success is supposed to look like. In Episode 18 of System Prompt, Peter and Val are joined by Jonny Havey of E-Learning Partners to discuss AI adoption, employee education, training ROI, and the role of clear KPIs. The conversation examines why training programs fail when they are disconnected from business outcomes and how organizations can measure whether AI education is actually improving performance. WHAT WE DISCUSS • Why buying AI tools does not guarantee adoption • The role of education in successful implementation • Why training must connect to business outcomes • How KPIs measure training effectiveness • Why ROI is broader than direct financial return • How AI is changing workplace education • Why employees need role-specific training • The difference between intelligence and wisdom KEY TAKEAWAYS AI ADOPTION REQUIRES MORE THAN ACCESS Giving employees an AI license does not mean they will use it effectively. They need to understand which problems it can solve, when it should be used, what data is appropriate to provide, and how outputs should be evaluated. TRAINING NEEDS CLEAR KPIS A training program should be tied to a measurable outcome. That could include faster task completion, fewer errors, reduced onboarding time, higher adoption, stronger confidence, or improved customer experience. The KPI should be defined before the training is created. ROI IS NOT ONLY FINANCIAL Training may create value through saved time, reduced risk, stronger consistency, better retention, and improved decision-making. Those outcomes may affect revenue or cost later, even when the return is not immediately financial. TRAINING SHOULD ALIGN WITH ORGANIZATIONAL GOALS Generic AI workshops often fail because they are disconnected from the work employees actually perform. Different teams have different workflows, risks, and responsibilities. Training becomes more useful when it is built around real business processes and expected outcomes. AI SHOULD SUPPORT WISDOM, NOT JUST OUTPUT AI can generate information quickly, but information alone does not create good judgment. Employees still need the experience and context required to evaluate recommendations and understand consequences. CHAPTERS 00:00 — The Importance of ROI and KPIs 05:16 — Aligning Training Programs with KPIs 08:36 — The Evolution of E-Learning Partners 24:19 — The Impact of AI Adoption on Education 45:42 — The Role of E-Learning Partners ABOUT JONNY HAVEY Jonny Havey is the co-founder of E-Learning Partners and host of the Learning Transformed podcast. https://elearningpartners.com WATCH THE EPISODE https://youtu.be/M3iRj99gy7Y ABOUT SYSTEM PROMPT System Prompt covers AI infrastructure, automation, agents, enterprise platforms, training, and practical implementation.

  8. Jul 8

    How AI is Transforming Food Discovery with Stupid Good AI

    READ THE FULL EPISODE PAGE https://devmesh.tech/podcast/food-discovery-stupid-good-ai Restaurant discovery is still dominated by ratings, generic lists, and search results that often fail to reflect what a person actually wants. In Episode 17 of System Prompt, Peter and Val are joined by David Leibner, founder of Stupid Good AI, to discuss how artificial intelligence could create a more personalized way to discover food, restaurants, and local experiences. The conversation explores how Stupid Good AI combines community feedback, data science, and AI to move beyond static reviews and broad “best of” lists. David also discusses building a vertical AI product, the changing software-development process, open models, and how user behavior can improve recommendations over time. WHAT WE DISCUSS • Why restaurant discovery still feels broken • How AI can personalize food recommendations • Why ratings alone do not capture individual preferences • How community feedback improves discovery • Building a vertical AI product around a specific problem • The role of data science and open models • How AI is changing software development • Turning local restaurant data into useful recommendations KEY TAKEAWAYS FOOD DISCOVERY NEEDS MORE CONTEXT A highly rated restaurant is not automatically the right recommendation for every person. Useful discovery should consider taste, location, occasion, dietary needs, budget, atmosphere, and previous preferences. COMMUNITY DATA CAN IMPROVE RECOMMENDATIONS User feedback gives the system more than a single star rating. Over time, patterns across reviews, preferences, and behavior can help produce recommendations that are more relevant to each user. VERTICAL AI PRODUCTS CAN GO DEEPER Stupid Good AI focuses on one specific problem rather than trying to become a general-purpose assistant. That narrower scope creates an opportunity to build better data, workflows, and experiences around food discovery. AI DOES NOT REPLACE PRODUCT THINKING Models can support recommendations, classification, search, and personalization. The product still needs strong data, useful interfaces, reliable feedback loops, and a clear understanding of the user’s problem. ABOUT DAVID LEIBNER David Leibner is the founder of Stupid Good AI, a platform focused on improving how people discover restaurants and local food experiences. Learn more: https://stupidgood.ai Early-access users can mention System Prompt when contacting the Stupid Good AI team. ABOUT SYSTEM PROMPT System Prompt covers AI infrastructure, automation, agents, local models, enterprise platforms, and practical implementation.

See All (24)

About

System Prompt is a podcast about what’s actually happening in AI. Not hype. Not surface-level takes. We break down how AI is changing software, SaaS, infrastructure, and the way systems are built focusing on real-world tradeoffs, architecture decisions, and where the value is actually shifting. If you’re building, deploying, or thinking seriously about AI, this is for you.

Information

  • Creator
    Peter
  • Years Active
    2026
  • Episodes
    24
  • Rating
    Explicit
  • Copyright
    © 2026 Peter
  • Show Website
    System Prompt

You Might Also Like