DEV

Eric Lamanna

Software and AI development podcast. We cover all things software development, including today's advanced AI development tricks and techniques.

  1. 4 小時前

    Real-Time ML Inference: Wrangling Kafka and TensorFlow Serving

    Real-time machine learning inference has moved from a competitive advantage to a baseline expectation. This episode of Development tackles the architectural and operational realities of building a streaming inference pipeline — one that can ingest live events, score them against an ML model, and return predictions in milliseconds. The discussion is grounded in the deep-dive on streaming ML inference with Kafka and TensorFlow Serving and covers everything from initial design decisions to the production pain points that only reveal themselves under real traffic. Here's what this episode walks through: Why real-time inference matters: Use cases like fraud detection, content recommendation, and autonomous systems illustrate why latency measured in minutes — or even seconds — is simply no longer acceptable.Kafka as the data backbone: How Kafka's high-throughput, fault-tolerant, and replayable event streaming model keeps a continuous flow of fresh feature data moving to the model — no waiting for a batch to assemble.TensorFlow Serving in production: Why Google's model-serving system goes beyond simple hosting, offering version management, live rollouts, and both REST and gRPC interfaces for flexible integration.Kafka configuration trade-offs: Practical guidance on partition count, message retention, schema registries, and monitoring consumer lag — the decisions that trip up teams early and quietly.Scaling and optimizing TensorFlow Serving: When to scale horizontally via Kubernetes, when to optimize the model itself (quantization, distillation, request batching), and why GPU acceleration isn't always the right first move.Resilience and observability: Building error handling that prevents cascading failures — retry caps, circuit breakers, graceful degradation — alongside a monitoring stack (Prometheus and Grafana) that tracks the metrics that actually matter.The episode closes with an honest framing: combining Kafka and TensorFlow Serving is less a one-time technology choice and more an ongoing operational commitment. The teams that get the most out of this stack are the ones who invest in understanding it deeply, tune it deliberately, and treat observability as a first-class concern from day one. More from the show: if you enjoyed this episode, check out Why Deploying LLMs on Serverless Is a Beautiful Disaster for another candid look at the operational complexity hiding inside modern ML deployments. DEV

  2. 1 天前

    Why Deploying LLMs on Serverless Is a Beautiful Disaster

    Serverless computing promises infinite scale, zero infrastructure management, and pay-per-use economics — so the idea of running a large language model on top of it is understandably tempting. But the gap between that pitch and production reality turns out to be enormous. This episode of Development unpacks the very real architectural friction explored in this deep-dive on deploying large language models in serverless environments, walking through the core challenges and the hybrid strategies that actually hold up at scale. The conversation covers a lot of ground for engineers weighing up this architectural choice: Cold starts as a dealbreaker: When a serverless function wakes from idle, loading a multi-gigabyte LLM before processing a single token introduces multi-second delays that make real-time applications essentially non-functional — and "warm instance" workarounds quietly abandon the serverless cost model entirely.The statelessness mismatch: Serverless is built around clean-slate, stateless invocations, while LLMs depend on persistent conversational context. Offloading state to external caches or databases solves this in theory, but adds latency, new failure modes, and significant debugging complexity.Hard compute ceilings: Platform limits like AWS Lambda's 10 GB RAM and 15-minute execution cap are nowhere near sufficient for frontier-scale models. Teams are forced to choose between aggressive quantization or offloading inference to GPU-backed services — either way, the pure serverless architecture doesn't survive contact with the model.Storage and loading overhead: With no persistent in-memory model between invocations, every cold start may require downloading gigabytes from object storage, while persistent file system alternatives add their own cost and complexity.Security and isolation concerns: Ephemeral function instances handling sensitive user data, pulling open-source dependencies at runtime, and processing prompts vulnerable to injection attacks create a security surface that demands far more discipline than serverless's simplicity tends to encourage.The cost trap: Per-invocation billing sounds economical until LLM request volumes scale up. Compute-heavy, slow-generating models can turn a seemingly lean architecture into a budget crisis — making the hybrid model not just pragmatic, but financially necessary.The episode lands on a clear verdict: the teams that succeed with AI infrastructure aren't the ones chasing architectural purity. A hybrid approach — serverless handling lightweight orchestration, routing, and preprocessing, while dedicated GPU-backed infrastructure takes on serious inference workloads — consistently outperforms attempts at going fully serverless with a large model. Before committing to any LLM deployment strategy, it's worth pressure-testing the cold start behaviour, the context storage plan, and the cost model at realistic request volumes. For more on the NLP skills that underpin effective language model pipelines, check out the Development episode on Custom Tokenization Pipelines: The NLP Skill You Can't Afford to Skip. DEV

  3. 2 天前

    The Future of Coding Is No Coding at All — And Why Developers Need to Adapt Now

    The rise of no-code and low-code platforms isn't a passing trend — it's a structural change in how software gets built. This episode of Development draws on the article exploring why coders need to adapt to a no-code world to examine what this shift actually means for working developers: not the end of the profession, but a fundamental redefinition of where developer value lives. Hosts walk through the economic logic behind the no-code movement, the tools reshaping production workflows, and the specific skills that will separate indispensable developers from those who get left behind. Key topics include: Why the economics have already shifted — businesses pay for outcomes, not effort, and no-code platforms are changing what outcomes cost to deliver.The 80/20 reality of no-code capability — visual platforms can handle the majority of typical application needs, but custom logic, security architecture, and complex integrations still demand human expertise.Tools worth knowing now — Cursor's AI-assisted development environment and V0.dev's prompt-to-component workflow are highlighted as practical entry points for developers looking to expand their toolkit.The developer-as-strategist role — the roles shrinking are those built around generating boilerplate; the roles growing are those that combine technical intuition with product thinking and business fluency.Skills that remain non-negotiable — systems thinking, API and integration knowledge, and data security expertise don't disappear when drag-and-drop tools enter the picture.The mindset shift that matters most — moving from "how much code can I write?" to "what's the best way to solve this problem with the tools available?" is framed as the single most important career adaptation.The episode closes with a clear-eyed look at what hybrid development teams will look like — and why the developers who can move fluidly between visual builders and complex backend systems will be the ones shaping what gets built next. More from the show: if you're thinking about the technical skills that set developers apart, don't miss Custom Tokenization Pipelines: The NLP Skill You Can't Afford to Skip. DEV

  4. 4 天前

    Custom Tokenization Pipelines: The NLP Skill You Can't Afford to Skip

    Tokenization is the step most NLP developers treat as an afterthought — until their model starts mangling stock tickers, shredding hashtags, and treating multi-word entities like random word salad. This episode of Development makes the case that tokenization is a foundational design decision, not a checkbox, drawing on this in-depth guide to building custom tokenization pipelines for NLP models. If your model's behavior has ever felt inexplicably broken despite clean-looking data, the tokenizer is almost certainly where the story starts. The episode walks through the full landscape of tokenization approaches and explains why knowing the trade-offs — not just the defaults — is what separates functional NLP projects from fragile ones. Here's what's covered: Why whitespace tokenization fails at scale — languages without clear word boundaries, contractions, punctuation, emojis, and multilingual text all expose its limits almost immediately.Character-level tokenization — eliminates unknown-word problems entirely but produces sequences so long they become computationally punishing for transformer architectures.Rule-based and regex tokenization — powerful for structured, predictable text and extensible via tools like spaCy, but every edge case demands a new rule, and the edge cases never stop arriving.Subword tokenization (BPE, WordPiece, Unigram) — the backbone of modern large language models, handling out-of-vocabulary terms gracefully by breaking unfamiliar words into recognizable, reusable units.The "IKEA furniture" problem with pre-trained tokenizers — general-purpose tokenizers work until they meet domain-specific text (finance, medicine, social media), at which point their assumptions become liabilities.Performance at scale — why tokenization becomes a pipeline bottleneck long before most developers expect it to, and how tools like Hugging Face's Rust-powered tokenizers library help close the gap between speed and accuracy.The episode closes with a candid take: building a custom tokenization pipeline is genuinely difficult, unglamorous work — but getting it right pays dividends across everything downstream, from training speed to real-world generalization. For more on using AI to reduce tedious manual work in development workflows, check out the Development episode Stop Writing API Docs by Hand — Let AI Do the First Draft. DEV

  5. 5 天前

    Stop Writing API Docs by Hand — Let AI Do the First Draft

    Every developer knows the feeling: the code ships clean, the tests pass, and then the documentation tab sits open, untouched, for days. API docs have a way of drifting out of sync with the codebase almost immediately after they're written — and the real cost isn't inconvenience, it's the compounding miscommunication that erodes team trust over time. This episode of Development explores how large language models are changing that dynamic by handling the grunt work of the first draft, drawing on this in-depth look at automating API documentation with AI. Here's what the episode covers: Why manual docs fail predictably — not from carelessness, but because code and documentation live in separate places and move at different speeds, turning docs into archaeological artifacts.How LLMs read your code — using pattern-matching across naming conventions, function signatures, and inline comments to infer intent and generate structured descriptions at scale.The "junior dev first draft" mental model — AI output isn't perfect, but it's evaluated against the real alternative: no docs, outdated docs, or docs that took hours to write and were stale by Friday.Continuous documentation via CI/CD integration — triggering regeneration on every merge or push so that docs become a natural byproduct of development, not a painful batch project.Getting better results from any tool — the outsized impact of descriptive naming conventions, having a team style guide for review passes, and knowing which domain-specific logic still needs a human touch.Honest cost considerations — weighing subscription-based platforms against growing open-source options, and how to think about ROI when developer time is the real constraint.The core argument isn't that AI replaces developer judgment — it's that it moves the developer further up the loop, out of the tedious parts and into the decisions that actually require expertise. More from the show: if you're interested in building AI systems from the ground up, check out Training a Diffusion Model from Scratch: A Developer's Real Guide. DEV

  6. 6 天前

    Training a Diffusion Model from Scratch: A Developer's Real Guide

    Most developers who work with generative AI stop at the API layer — and that's fine, until curiosity kicks in. This episode of Development pulls back the curtain on what it genuinely takes to train a diffusion model from the ground up, drawing on the step-by-step guide to training a diffusion model for custom image generation published at DEV. Whether the goal is a specialized creative tool, a proprietary image pipeline, or simply a deeper understanding of how these systems work, this episode treats the topic with the seriousness it deserves — no hand-waving, no skipped steps. Here's what the episode covers: How diffusion models actually work — the intuition behind learning to reverse a noise process, and why that iterative denoising approach produces such high-quality outputs.Data curation as a first-class concern — why the visual distribution of your training set directly shapes what your model can and can't generate, and what "good enough" data actually looks like in practice.Hardware and environment setup — the real GPU requirements, cloud provider options, and why adding an experiment-tracking tool like Weights and Biases from day one saves significant pain later.Architecture choices and the U-Net backbone — what makes U-Nets well suited to denoising tasks, and why starting from an existing open-source implementation beats building from absolute zero.Reading the training signals — how to interpret loss curves and early sample outputs, what normal early-stage fuzziness looks like versus a run that's genuinely broken, and how to troubleshoot common failure modes like overfitting, underfitting, and out-of-memory errors.Fine-tuning and deployment — how a shorter, focused training pass can specialize a general base model, and practical ways to wrap a finished model in a REST API, a local tool, or an interactive dashboard.The honest takeaway from this episode: training a diffusion model from scratch demands compute, patience, and careful iteration — but the reward isn't just a working model. It's a mechanistic understanding of generative AI that holds its value long after the surface-level tooling has moved on. For more on making deep learning models leaner without sacrificing what matters, check out the earlier episode Neural Network Quantization: Shrinking Models Without Losing Accuracy. DEV

  7. 7月21日

    Neural Network Quantization: Shrinking Models Without Losing Accuracy

    Deploying a well-trained machine learning model to a resource-constrained environment — a smartphone, an IoT sensor, or a cost-sensitive cloud setup — often reveals a painful gap between a model's theoretical requirements and what real hardware can deliver. This episode of Development tackles that gap head-on, exploring how neural network quantization makes production AI more practical across the board. The discussion is drawn from this in-depth article on reducing model size without losing accuracy, and goes further by walking through the tradeoffs developers actually face in the field. Here's what the episode covers: What quantization actually does: How converting model weights from 32-bit floating-point (FP32) to 8-bit integers can shrink a model to roughly a quarter of its original size — and why modern CPUs, mobile chipsets, and edge processors are built to take advantage of it.Why accuracy holds up better than expected: Neural networks are inherently redundant and resilient; in most image recognition and NLP tasks, the accuracy drop from FP32 to INT8 is a fraction of a percent — effectively invisible to end users.Post-training quantization vs. quantization-aware training (QAT): The classic convenience-vs-quality tradeoff — when to reach for each approach, and how frameworks like TensorFlow Lite and PyTorch support both.Mixed-precision quantization: A more surgical strategy that assigns higher bit-widths to sensitive layers (such as attention mechanisms in transformers) while aggressively compressing layers that tolerate lower precision.Where quantization delivers the most value: On-device AI for privacy and latency, ultra-constrained edge hardware where deployment feasibility is binary, and cloud serving where cutting memory use directly translates to lower infrastructure costs.Real pitfalls to avoid: Layer sensitivity, calibration shortcuts that cause range-clipping errors, and the hardware compatibility gaps that can prevent quantized models from hitting their theoretical performance ceilings.The episode closes with a practical starting framework: benchmark post-training quantization first, measure latency and memory — not just accuracy — and escalate to QAT or mixed-precision only when the simpler approach falls short. For more on the intersection of machine learning and developer tooling, check out the episode on AI-Powered Linting: Smarter Static Code Analysis With Machine Learning. DEV

  8. 7月20日

    AI-Powered Linting: Smarter Static Code Analysis With Machine Learning

    Traditional linters are reliable workhorses, but they can only flag what someone thought to write a rule for. This episode of Development explores what happens when you pair static code analysis with machine learning — moving beyond deterministic rule enforcement toward a tool that can recognize subtle, historically problematic patterns the way a seasoned developer does. The discussion is grounded in this deep-dive on building an AI-powered linter with ML models, and goes further into the practical decisions teams face when taking this approach seriously. Here's what the episode covers: Why traditional linters hit a ceiling: Rule-based analysis is only as strong as the rules themselves — edge cases, framework-specific vulnerabilities, and nuanced anti-patterns routinely slip through.How ML changes the analysis model: Instead of a checklist, a trained model learns the statistical shape of problematic code, surfacing issues that no explicit rule would catch.Model selection and data quality: Bigger isn't always better — a focused model trained on well-labeled, domain-specific examples often outperforms a general-purpose LLM for targeted linting tasks.The noise-to-signal problem: An AI linter that cries wolf too often gets ignored; rolling out high-confidence, high-stakes catches first (such as security vulnerabilities) is the key to earning team trust before expanding scope.Continuous improvement and feedback loops: Unlike a static ruleset, an ML-based linter drifts out of alignment if left untouched — developer overrides and missed bugs are valuable retraining signals.Transparent onboarding: Introducing the tool with honest expectations about early false positives turns healthy skepticism into buy-in rather than resistance.The episode makes a clear case that an AI-powered linter is best understood as an always-on first pass — not a replacement for human review, but a way to surface real problems earlier and cheaper than any runtime error ever could. If you're interested in where the intersection of AI and edge hardware is heading next, check out the recent episode Edge AI Explained: Running Smarter Models on Tiny Devices for a complementary look at deploying ML in constrained environments. DEV

顯示全部 (54)

簡介

Software and AI development podcast. We cover all things software development, including today's advanced AI development tricks and techniques.

資訊

  • 創作者
    Eric Lamanna
  • 活躍年代
    2026年
  • 集數
    54
  • 年齡分級
    兒少適宜
  • 版權
    © 2026 DEV.co
  • 節目網站
    DEV