Embedded AI Podcast

Embedded AI Podcast

A podcast about using AI in embedded systems -- either as part of your product, or during development.

  1. 7 Aug

    E21 AI security research and AI security

    Ryan and Luca tackle the hot topic of AI-driven security research, sparked by the release (and brief containment) of Anthropic's Mythos tool. Ryan, drawing on his 20+ years in cybersecurity, delivers a surprisingly reassuring message: if you've been doing proper engineering, AI-found vulnerabilities aren't the existential threat the hype suggests. We explore how Mythos and similar tools are flooding the CVE database with thousands of new vulnerabilities, but discuss why this doesn't automatically mean more successful attacks. Ryan explains the crucial difference between finding a bug and actually exploiting it, and why embedded systems developers shouldn't panic—but should definitely have their update processes sorted. The conversation covers everything from botnet refrigerators to Ukrainian security cameras, threat modeling with AI assistance, and why AI makes such a relentlessly effective hacker (spoiler: it doesn't get bored). Bottom line: the weapons haven't changed, you just need to install the bulletproof glass you should have had all along. Key Topics: [02:30] Introducing Mythos: AI tool for finding software vulnerabilities, its May release, US containment, and recent re-release[05:15] The flood of AI-generated bug reports: distinguishing real vulnerabilities from noise, and the burden on maintainers[08:45] Mythos by the numbers: 6,000+ critical vulnerabilities found, 90% true positive rate, but does it matter for your system?[12:00] The reachability problem: having a bug vs. being able to exploit it, and why solid engineering processes matter more than bug counts[16:30] Embedded systems challenges: BSP version conflicts, regulatory approval nightmares, and the CRA/FDA compliance push[21:00] No uptick in actual attacks: why more CVEs doesn't equal more breaches, and what motivates attackers (hint: money, not bugs)[28:45] Embedded systems' blessing and curse: air-gapped devices vs. internet-connected vulnerabilities, and the botnet refrigerator story[35:20] Using AI for defense: network analysis, threat modeling, traffic pattern recognition, and why AI is the best grep you've ever seen[42:00] AI as the relentless attacker: no social norms, no contracts, just pure problem-solving—and why that's both powerful and concerning[47:30] The offensive vs. defensive mindset: why AI bridges both motivational patterns and what that means for security teamsNotable Quotes: "Just because there's more CVEs doesn't really change the defensive posturing that you need to do. If you have a solid engineering process to provide updates to your system and test and verify those updates actually work, it doesn't matter how many bugs you find. You can find two. You can find 6,000." — Ryan Torvik "AI doesn't have social norms. AI will just go and do things. And yes, there are guardrails that they're trying to put on, but like, I don't know if you've seen AI just ignore parts of your prompt before." — Ryan Torvik "There's not a death star. This is not an existential crisis. This is not something new. It changes the game but not in a way that we can't handle. The weapons haven't changed, you just need to install the bulletproof glass." — Ryan Torvik Resources Mentioned: Anthropic's Mythos - AI tool for automated vulnerability discovery in software, released in May 2024, briefly restricted by US authorities, then re-releasedCVE Database - Common Vulnerabilities and Exposures database, currently experiencing significant growth due to AI-assisted vulnerability discoveryAgile Embedded Podcast Slack - Community Slack channel where Ryan participates, sister podcast to Embedded AITulip Tree Tech - Ryan's company focused on improving embedded development processes, including emulator-in-the-loop solutionsluca.engineer - Luca's website with links to training courses, LinkedIn, and other professional activities

  2. 24 Jul

    E20: "AI Will Take My Job :-( " - Why Your Job Is (Probably) Safe

    We tackle the elephant in the room: will AI take our jobs? Spoiler alert - probably not, but your job will definitely change shape. Ryan and Luca dig into what software development actually is (hint: it's not just mashing keyboards), why embedded systems might be particularly safe from AI disruption, and what we can learn from the Luddites and steam engines. Drawing on Luca's experience with DevOps transformations and Ryan's work running a company, we explore why engineers keep solving the wrong problems, what Black & Decker actually sells (it's not drill bits), and why vibe-coding your email server is a terrible idea. The real question isn't whether AI will replace you - it's whether you understand what your actual job is in the first place. Plus: Pokemon evolution as career advice, and why AI is the new Agile fairy dust. Key Topics: [02:30] The real job vs. the mechanical task - why writing code isn't actually your job[08:45] Why embedded systems are particularly safe from AI disruption - complexity, hardware interaction, and undocumented quirks[15:20] The Black & Decker lesson: selling holes in walls, not drill bits - understanding what customers actually want[22:10] Historical parallels: steam engines, DevOps, and why demand grows faster than automation[28:40] The reality check: most developers aren't using AI systematically yet - you're not behind the curve[32:15] AI as the new Agile fairy dust - why magic solutions never work without process and understandingNotable Quotes: "Your job is not writing code. Your job is making product. If you think about the Luddites, their job was not operating a loom. Their job was making clothes." — Ryan Torvik "Writing code is really the smallest part of software development. Most of it is sitting in front of the screen, looking up and to the left, and figuring out what code to write." — Luca Ingianni "The CEO of Black & Decker was once quoted as saying: We are not in the business of selling drill bits. We are in the business of selling holes in the wall. If we had laser cannons that made holes in walls, people would buy the laser cannons instead." — Luca Ingianni Resources Mentioned: Agile Embedded Podcast Slack - Community discussion channel where you can reach Ryan and Luca, with a dedicated sub-channel for Embedded AI topicsLuca's Website - Contact Luca for consulting on using AI effectively in embedded systems contexts - multiple contact options available

  3. 10 Jul

    E19: SDD frameworks, and WhittleSpec - Rethinking AI-Assisted Development with Feedback Loops

    Luca unveils WhittleSpec, his new open-source framework for AI-assisted development that challenges the waterfall assumptions baked into most coding tools. Born from frustration with frameworks like SpecKit that treat specifications as static documents, WhittleSpec emphasizes continuous learning through retros, refinement, and test-driven development. We explore why most AI coding frameworks make typing faster but don't help with the hard part—the thinking—and discuss how proper feedback loops and vertical slicing can lead to more trustworthy software. The conversation ranges from the philosophy of whittling away what doesn't fit (versus plowing ahead blindly) to practical implementation details: specification, planning, task breakdown, TDD cycles, and retrospectives. Luca argues that professional software engineering requires systematic processes that support learning at every step, not just tools that generate code quickly. We also touch on the missing pieces in current frameworks: support for safety-critical development, long-term roadmaps, and embedded systems considerations. Key Topics: [02:30] Introducing WhittleSpec and the problem with current AI coding frameworks[08:45] Why AI tools make the easy part easier but leave the hard part hard[15:20] The waterfall trap: static specifications vs. living documents[22:10] Mapping the landscape of AI development frameworks (SpecKit, BMAT, Kiro, etc.)[28:40] How WhittleSpec works: decide, specify, plan, tasks, and TDD cycles[38:15] The critical role of retrospectives and the 'refine' skill for course correction[45:30] Vertical slicing vs. layer-by-layer implementation: tracing bullets through the stack[51:00] Missing pieces: safety-critical development, long-term roadmaps, and embedded considerationsNotable Quotes: "No engineer ever said, 'I wish I could type curly brackets faster.' That was never quite the bottleneck. The hard part is sitting in front of your screen and going, 'hmm.'" — Luca Ingianni "SpecKit is just plain old horrendous waterfall. There are no provisions at all in it for learning. The idea is you specify something well enough and then you just walk away, sip a coffee, the machine does its thing. That approach has never ever worked." — Luca Ingianni "Your initial specification is not going to be right. As you implement the actual solution, you're going to learn things that's going to change what you need to accomplish with the spec. You might need to change completely what your expectations are." — Ryan Torvik Resources Mentioned: WhittleSpec - Luca's new open-source AI-assisted development framework emphasizing feedback loops, TDD, and iterative refinementSpecKit - GitHub's AI coding framework discussed as an example of waterfall-style specification-driven developmentAgile Embedded Slack - Community Slack channel now open to Embedded AI podcast listeners for discussion and questionsLuca.engineer - Luca's website with links to all his projects and ways to reach himTulipTreeTech - Ryan's company working on AI-generated models for pre-silicon firmware validation

  4. 26 Jun

    E18: Shawn Hymel on Edge AI - NPUs, Deployment Challenges, and the Future of Embedded ML

    Ryan and Luca sit down with Shawn Hymel, an educator and course creator focused on edge AI and embedded systems. We explore what's changed in the past few years—from basic keyword spotting to full object detection on microcontrollers, thanks to integrated NPUs. Shawn walks us through the messy reality of deploying ML models to embedded hardware: vendor-specific toolchains, dependency hell, and the ongoing challenge of making edge AI accessible. We discuss what students and experienced engineers need to learn (or unlearn) to work effectively in this space, and look ahead to exciting developments like reinforcement learning on tiny devices and neuromorphic computing. It's a candid, technical conversation about where edge AI stands today and where it's headed. Key Topics: [03:30] Why run ML on microcontrollers? Power, size, and application-specific advantages[06:00] Keyword spotting as the original killer app for edge AI[08:45] The game-changer: NPUs enabling full object detection on microcontrollers[13:20] Privacy benefits of on-device processing vs. cloud-based inference[16:00] How NPUs work under the hood and the vendor-specific deployment reality[24:30] The painful parts: dependency hell, graph compilers, and memory arena sizing[29:00] Using AI tools (LLMs) to navigate vendor documentation and generate code[33:45] What CS students and ECE students each need to learn for edge AI[40:15] Shifts in university enrollment: ECE rising, CS declining[44:00] When you don't need AI: PID loops and deterministic solutions still matter[47:30] Looking ahead: reinforcement learning on microcontrollers and neuromorphic computingNotable Quotes: "Five, six years ago, we didn't have full object detection on microcontrollers. Now with NPUs, we can do things like full YOLO on a 320x320 image—milliwatts of power, full object detection. That was not a thing five years ago." — Shawn Hymel "Expect to spend a day or two getting inference to actually run. The docs are still new, the graph compilers are fairly new. You're going to end up in dependency hell—both on the Python side and when you bring it over to the embedded side." — Shawn Hymel "If a PID loop solves your need, there is absolutely no reason to put AI on there. That is a solved problem. Don't do it—just use a PID loop." — Shawn Hymel Resources Mentioned: Shawn Hymel's Website - Shawn's main site with links to free and paid courses on edge AI and embedded systemsOpenMV - Computer vision platform for microcontrollers, including the new AE3 board with NPU supportEdge Impulse - Platform that simplifies ML model deployment to embedded devices, supporting various NPUsAndrew Ng's Coursera ML Course - Foundational machine learning course recommended for understanding the math behind MLTensorFlow Lite for Microcontrollers (LiteRT) - Framework for running ML models on microcontrollers across different platforms

  5. 12 Jun

    E17: Switching Providers - Insulating Yourself from AI Vendor Lock-in

    Ryan and Luca tackle a challenge many AI users are facing: what happens when your AI provider starts acting up? Drawing from recent experiences with Anthropic's capacity issues, secret billing practices, and model degradation, we explore practical strategies for avoiding vendor lock-in. We discuss the three layers of complexity: the model itself, the harness (like Claude Code or GitHub Copilot), and your authored content (skills, MCP servers, prompts). Each layer presents different challenges when switching providers. Ryan shares his approach of stepping back to simpler, more granular prompting to stay provider-agnostic, while Luca experiments with maintaining escape hatches to other platforms. We also look at the realities of running local models and the tradeoffs between convenience and control. The bottom line? Pick one system, get proficient, but prepare your exit strategy - because in this volatile landscape, you'll likely need it sooner than you think. Key Topics: [02:30] Anthropic's recent troubles: capacity issues, model degradation, and gaslighting users[06:45] The Hermes.md billing scandal - secret charges for having a specific filename[10:20] Ryan's approach: stepping back to simpler, granular prompting for provider independence[15:00] The three layers of complexity: model, harness, and authored content[18:30] Why models aren't interchangeable - different flavors, tokenizers, and caching strategies[24:15] Luca's tone-of-voice challenge: getting consistent writing style across models[30:00] Running local models and private inference as alternatives to frontier models[35:45] Practical strategies: maintaining escape hatches without parallel systems[40:20] Luca's solution: versioning authored content separately with symlinksNotable Quotes: "The more you actually make use of AI in your work, the more you use it as a force multiplier, the more painful it becomes if that force multiplier goes away." — Luca "It's about total clock time. If you get a one shot and then have to redo it again, how much of that clock time is being used effectively?" — Ryan "Pick one, stick with it, be proficient in it. But prepare yourself to have to escape eventually, because the situation is so volatile." — Luca Resources Mentioned: Claude Code - Anthropic's AI coding assistant with hooks and skills supportOpenCode - Provider-agnostic AI coding harness that supports multiple modelsGitHub Copilot CLI - Multi-provider AI assistant with dropdown model selectionMCP (Model Context Protocol) - Protocol for extending AI capabilities across different harnessesOllama - Tool for running local AI models on your own hardware

  6. 29 May

    E16 Running LLMs Locally: Privacy, Performance, and Practical Trade-offs

    We explore what it really means to run AI models locally instead of relying on cloud providers like OpenAI or Anthropic. From powerful desktop setups with dual NVIDIA RTX 3090s to tiny models running on embedded systems, we cover the full spectrum of local AI deployment. Luca shares hands-on experience running local models for client work, explaining the hardware requirements (spoiler: you need fast VRAM, not just lots of RAM), performance trade-offs, and practical tools like Ollama and LM Studio. We discuss how modern open-weight models from Meta, Google, and Chinese companies compare to hosted solutions - typically about a year behind state-of-the-art but surprisingly capable. We also look at edge AI applications, from elderly fall detection to traffic accident monitoring, where compact models shine. The conversation covers context window limitations, quantization techniques, and why getting started is easier than you might think - though you'll need to manage expectations about what local models can deliver compared to their cloud-based cousins. Key Topics: [00:00] Introduction: What are local models and why run them?[02:30] Hardware requirements: VRAM vs system RAM, and why graphics cards matter[05:45] Luca's setup: dual RTX 3090s and real-world client work with local models[08:20] Performance metrics: time to first token, tokens per second, and output quality[12:00] Tiny models for edge AI: Google's 270M parameter model and specific use cases[15:30] Tools and workflows: Ollama, LM Studio, and OpenAI-compatible APIs[18:45] Where models come from: Hugging Face, Meta's Llama, and the open-weight ecosystem[22:10] Context window limitations and quantization techniques[25:00] Getting started: realistic expectations and practical first stepsNotable Quotes: "What those models really need is tons of memory and as fast memory as you can get it. This is why people like Macs because they've got the unified memory architecture." — Luca Ingianni "Modern models are actually pretty good. They are smaller, so they will have less knowledge. They are a tad slower. They struggle with much smaller context windows. But if you can work within those bounds, they work pretty well." — Luca Ingianni "Using AI is a different mindset. It's a different way of thinking about solving a problem, but it's just another tool in your toolbox and it's just another way of getting work done." — Ryan Torvik Resources Mentioned: Ollama - Docker-like tool for running local LLMs with simple pull/run commands and OpenAI-compatible APILM Studio - Graphical user interface for downloading and running local models easilyOpen WebUI - Web interface for local models that mimics ChatGPT's chat interfaceHugging Face - Repository with hundreds of thousands of models in various sizes and configurationsMeta Llama - Open-weight model family from Meta that helped start the local LLM movementGoogle Gemma - Model family from Google including compact vision-capable models (270M parameters)

  7. 15 May

    E15 Julia Imlauer on introducing AI into organizations

    We talk with Julia Imlauer, Director for AI Strategy and Development, about the practical realities of introducing Gen AI tools into engineering organizations. Julia shares her experience training engineers and management across multiple companies, revealing what actually works—and what doesn't—when rolling out LLM-based coding assistants. The conversation covers the adoption curve from early enthusiasts to skeptics, the importance of giving engineers dedicated time to experiment, and why "word of mouth" matters more than top-down mandates. We discuss the messy reality of legacy codebases where LLMs shine, the challenge of managing AI-generated code in pull requests, and the emerging cost crisis as providers shift from subsidized to consumption-based pricing. Julia also addresses common fears about junior developer training and the misconception that LLMs can replace human expertise. Key Topics: [02:30] Julia's background in robotics and machine learning, and how she started introducing Gen AI tools into development workflows[05:15] Why LLMs help with messy, historically-grown codebases more than pristine template-based workflows[09:45] Different developer preferences: IDE integration vs. separate LLM interfaces[13:20] The adoption curve: early adopters, skeptics, and the crucial middle mass that needs convincing[16:00] Training sessions that work: giving engineers blocked time to experiment, not step-by-step tutorials[19:30] The power of word-of-mouth and why ambassador programs sometimes help spread adoption[24:10] Training management and executives: IP protection, managing AI-generated code in pull requests, and supporting skeptical engineers[29:45] The billing nightmare: token consumption, opacity in pricing models, and the shift away from all-you-can-eat plans[35:20] Common skeptic concerns: fear of replacement, adoption pain, and outdated first impressions from early LLM versions[38:50] Junior developer training with LLMs: why pair programming and code reviews still matter more than writing every line by hand[43:15] Communication breakdowns between departments: translating vocabulary between IT, legal, and engineering[46:30] Looking ahead: the need for agentic frameworks, standardization, and routing models to manage costsNotable Quotes: "What they do for you is really, they free your engineers to focus on things that matter. They write the boilerplate code. They write the code which you don't want to think about, which is a necessity, which you need to have to make everything work. But it's just like a doing concept." — Julia Imlauer "It sounds stupid when you say you need kind of a education training session for engineers to use this tool, but exactly this is what I experienced—they don't need you know a training in the ways of you need to click here and here and here. What they need is blocked two hours out of their daily coding hustle and really get kind of a free mind to take a look at the tool." — Julia Imlauer "There is this misconception of, okay, we don't need junior developers anymore. They just cost and then an LLM can do that. And let's just pile more on our senior developers. I mean, this is not working. And everyone who is in software development understands that." — Julia Imlauer Resources Mentioned: Caveman - A GitHub repo that compresses prompts to very basic words to reduce context window size and token consumptionOpen Claw / Nemo Claw - Open source agentic framework projects exploring standardized interfaces for LLM agentsMCP (Model Context Protocol) - An emerging standard for LLM tool integration mentioned as a potential path toward provider-agnostic frameworks

  8. 1 May

    E14 Kwabena Ageyman on OpenMV

    We sit down with Kwabena Ageyman, co-founder of OpenMV, to explore how microcontrollers have evolved from simple 8-bit chips to AI-capable systems that rival desktop computers. Kwabena walks us through OpenMV's journey from the CMU Cam days to their latest products—the OpenMV Cam AE3 and N6—which pack neural network accelerators, image signal processors, and H.264 encoders into single-chip packages. What makes these systems remarkable isn't just raw performance (250+ gigaops for AI inference), but what they enable: battery-powered computer vision deployments with no infrastructure requirements. Kwabena demonstrates running a complete web server with live video streaming—all in MicroPython on a microcontroller. We discuss the practical implications: doorbell cameras that don't phone home, parking lot monitors that run on solar panels, and industrial vision systems that don't require conduit runs. The conversation touches on hard technical choices (why debayer images even for AI?), the underappreciated value of MicroPython for complex applications, and the infrastructure costs that kill many promising AI deployments. Kwabena also previews what's coming: transformer support on microcontrollers and WiFi HaLow for long-range, high-bandwidth connectivity. For anyone working on edge AI or embedded vision, this episode offers both practical insights and a glimpse of what's possible when hardware acceleration meets thoughtful software design. Key Topics: [00:03] Introduction to OpenMV and the evolution from "impossible" computer vision on microcontrollers to AI-capable systems[00:05] Technical specs of the AE3 and N6: 250 gigaops performance, 5-64MB RAM, image signal processors, and H.264 encoding[00:12] Why image processing steps like debayering matter even for AI applications[00:18] The infrastructure cost problem: why power, connectivity, and deployment logistics kill many AI projects[00:28] Running MicroPython on microcontrollers: web servers, RTSP streaming, and complex applications without Linux[00:35] Live demo: complete web interface with video streaming running on a microcontroller[00:42] Real-world use cases: YOLO object detection, face tracking, drowsiness detection, and parking lot monitoring[00:52] The future: transformer support on microcontrollers and WiFi HaLow for long-range connectivityNotable Quotes: "Ten years ago when we started, if you Googled for computer vision on microcontrollers, you got a single Stack Overflow reply about how that was impossible. Since then, a lot has changed." — Kwabena Ageyman "The product dies when you have to tell people: I want you to put 10,000 of these in the field. They look at the infrastructure cost and say, what does that look like end to end? Is that actually going to be a net benefit, or is it just sexy and looks cool?" — Kwabena Ageyman "Having AI on the edge actually unlocks privacy. It is a decision to collect all the data and store it forever. If you have Edge AI locally on these devices, the device manufacturer can say: we're actually not going to go into a format where we have infinite data collection of everything." — Kwabena Ageyman Resources Mentioned: OpenMV - OpenMV's website with products, documentation, and community resourcesRoboflow - Cloud platform for training computer vision models, partnered with OpenMVEdge Impulse - Edge AI development platform, OpenMV partner for model trainingWiFi HaLow - Long-range, low-power WiFi technology (up to 10 miles) mentioned for future connectivityEmbedded Online Conference - Conference where Kwabena, Luca, and Ryan will be speaking

See All (21)

About

A podcast about using AI in embedded systems -- either as part of your product, or during development.

Information

  • Creator
    Embedded AI Podcast
  • Years Active
    2025 - 2026
  • Episodes
    21
  • Rating
    Clean
  • Copyright
    © Embedded AI Podcast
  • Show Website
    Embedded AI Podcast

You Might Also Like