System Prompt

Peter

System Prompt is a podcast about what’s actually happening in AI. Not hype. Not surface-level takes. We break down how AI is changing software, SaaS, infrastructure, and the way systems are built focusing on real-world tradeoffs, architecture decisions, and where the value is actually shifting. If you’re building, deploying, or thinking seriously about AI, this is for you.

  1. 2d ago

    The Secret Behind Every AI Application

    The conversation delves into the intricacies of model routing, emphasizing the importance of routing systems, types of routing, regular expression, tracing, deterministic routing, and incremental improvements. Key aspects of routing logic, silent reroute, and output quality are also explored, culminating in a comprehensive understanding of model routing and its impact on AI experiences. Takeaways Routing systems play a crucial role in determining the reliability, consistency, and security of AI products.Incremental improvements in routing contribute to the overall improvement of AI experiences. Chapters 00:00 Introduction to Model Routing08:14 Types of Routing Systems18:28 Tracing and Observability in Routing26:21 Key Aspects of Routing Logic

    33 min

  2. Jun 3

    AI Hardware Revolution

    In this episode, Val and Peter explore the future of AI workers, focusing on the impact of hardware on AI workloads and the shift from cloud-based to device-level AI processing. They discuss the NVIDIA DGX Spark, its features, the CUDA ecosystem, and the challenges it presents. Additionally, they compare the Apple M5 and NVIDIA RTX Spark laptops, highlighting the cost trade-off and use case for mid-sized businesses. Finally, they delve into the disruptive impact of AMD in the AI hardware market with the Strix Halo and Gorgon Halo. The conversation delves into the AMD ecosystem and inference, API costs, workflow optimization, small teams and local device optimization, metered inference and cost considerations, routing and gateway for inference, hardware investment at scale, AI leveraging, and cost analysis, as well as inference cost and capability. Takeaways The evolution of AI workers is influenced by hardware advancementsThe shift from cloud-based to device-level AI processing has significant implications for businesses AMD ecosystem and inference considerationsCost analysis and optimization for AI leveraging Chapters 00:00 The Future of AI Workers12:12 Apple M5 and NVIDIA RTX Spark Laptops21:10 AMD Strix Halo and Gorgon Halo26:12 Small Teams and Local Device Optimization33:25 Hardware Investment at Scale40:21 Inference Cost and Capability

    43 min

  3. May 27

    AI in regulated industries

    The conversation delves into the challenges and impact of AI in regulated industries, emphasizing the importance of doing what's right and unlocking value while balancing innovation and compliance. It explores the ethical and legal implications of AI, the risks of overestimating AI capability, and the impact of AI on legal processes. Additionally, it discusses the training and responsibility in AI, the role of junior employees in AI management, and the impact of AI on human-to-human interaction. Finally, it addresses the future of AI and legal responsibility, the impact of AI on legal discovery, and the role of Privileg in AI oversight, while balancing experimentation and legal oversight. The conversation delves into the transformative impact of AI in unlocking opportunities, addressing legal liability and model bias in fintech, the implications of government AI and regulation, the significance of Chat GPT, and rapid-fire Q&A on AI sectors, regulatory misconceptions, and advice for founders. Takeaways The importance of ethical and compliant AI implementationThe need for training and responsibility in AI usage AI's game-changing impact on opportunitiesLegal liability and model bias in fintech Chapters 00:00 AI in Regulated Industries06:36 Ethical and Legal Implications of AI16:27 The Future of AI and Legal Responsibility23:49 Unlocking Opportunities with AI33:00 Government AI and Regulation40:08 Chat GPT and Regulatory Implications

    47 min

  4. May 20

    Is AI Native hype?

    In this episode, Val and Peter discuss the concept of being AI native, exploring the challenges and misconceptions surrounding AI native builders and AI native products. They delve into the need for deterministic structures and processes in AI native products, the role of traditional software engineering practices, and the importance of planning and research in building AI native products. The conversation delves into the reality of building AI-native products and the role of AI in traditional systems. It emphasizes the importance of understanding the process and demystifying the sensationalism around AI-native products. Takeaways AI native products require deterministic structures and processes to ensure consistent and credible outputs.AI native builders leverage AI to accelerate product development while maintaining traditional software engineering practices. AI-native builders are more than just traditional software engineers and should be seen as systems architects.The term 'AI-native product' is more about marketing and sensationalism than a true representation of the product.

    47 min

  5. May 13

    AI in Business - Using the Right Tool for the Right Problem

    The conversation delves into the challenges and opportunities of leveraging AI in business, particularly in the context of inventory management and customer-facing chatbots. It emphasizes the importance of understanding the problem, ensuring that AI solutions provide more value than cost, and building trust and empathy in AI implementation. Takeaways Understanding the problem is crucialAI solutions should provide more value than costEmpathy and trust are essential in AI implementation Chapters 00:00 Leveraging AI in Business06:46 Point of Sale Systems vs. AI13:36 Tailored AI Solutions for Businesses19:18 Customer-Facing Chatbots30:20 Data Cleaning for AI Implementation

    39 min

  6. May 6

    Episode 8: Prompt Engineering vs RAG vs Finetuning

    The conversation covers the importance of prompt engineering, the role of prompting in AI model performance, the use of keyword search for refining AI outputs, and the introduction to Retrieval Augmented Generation (RAG) for further refinement. The conversation delves into the technical aspects of data storage, canonicalization, and the use of MariaDB for vector store and operational data. It emphasizes the importance of efficiency and cost considerations in refining RAG systems and the need for human involvement in AI models. The discussion also explores the purpose and benefits of fine-tuning AI models, an iterative approach to AI model development, scaling, system integration, and the future of AI technologies. Takeaways Prompting is crucial for AI model performanceKeyword search and RAG are important for refining AI outputs Canonicalization and normalization reduce the amount of embedded logs by 70%Fine-tuning AI models requires a clear understanding of the desired output and iterative testing Chapters 00:00 Introduction to Prompt Engineering07:15 Using Keyword Search13:00 Introduction to RAG24:59 Data Storage and Canonicalization33:10 Understanding Fine-Tuning of AI Models40:18 Iterative Approach to AI Model Development49:54 Edge Technologies and Future of AI

    51 min

  7. Apr 29

    Episode 7 : AI News Today

    The conversation covers the degradation of AI model quality, the impact of API costs, and the dynamics of competition and market trends in the AI industry. It delves into the challenges faced by companies like Anthropic and OpenAI, as well as the implications for enterprise users and the broader AI ecosystem.

    40 min

  8. Apr 22

    Episode 6: AI & Ethics

    The conversation delves into the ethical considerations of AI implementation, its impact on workplace productivity, and the reshaping of jobs. It also explores the role of AI in decision-making, critical thinking, and education. The need for responsible AI implementation and the importance of AI literacy and training are highlighted throughout the discussion. Takeaways Responsible AI implementationEthical considerations in AIImpact of AI on education and workplace productivity

    40 min
See All (13)

About

System Prompt is a podcast about what’s actually happening in AI. Not hype. Not surface-level takes. We break down how AI is changing software, SaaS, infrastructure, and the way systems are built focusing on real-world tradeoffs, architecture decisions, and where the value is actually shifting. If you’re building, deploying, or thinking seriously about AI, this is for you.

Information

  • Creator
    Peter
  • Years Active
    2K
  • Episodes
    13
  • Rating
    Explicit
  • Copyright
    © 2026 Peter
  • Show Website
    System Prompt