DEV

Eric Lamanna

Software and AI development podcast. We cover all things software development, including today's advanced AI development tricks and techniques.

  1. 13小时前

    Building a Custom AI Code Refactoring Tool With GPT-4-Turbo

    Technical debt doesn't clean itself — but what if a well-engineered AI tool could do most of the heavy lifting? This episode of Development examines the practical architecture behind a custom GPT-4-Turbo refactoring tool, drawing on this deep-dive article on building a custom AI code refactoring tool. It's a candid look at what it genuinely takes to turn a powerful language model into something trustworthy enough to run against a real codebase. The episode walks through four interconnected layers of the problem — goal definition, prompt engineering, pipeline architecture, and output validation — covering: Defining the scope precisely: "Make the code better" is not an instruction. Effective tools encode specific, parameterized goals — readability, style compliance, function decomposition — and explicitly lock down what the model is not allowed to change.Writing prompts with guardrails: Surgical prompts that name a target standard, restrict method signature changes, and ban new external dependencies dramatically reduce the chance of GPT-4-Turbo making confident, unwanted architectural decisions.Handling the token window: Because codebases exceed GPT-4-Turbo's context limit, intelligent segmentation is essential — keeping related functions together and tracking dependencies so refactored code doesn't break on integration.Building a resilient API pipeline: Rate limits, quota overruns, and transient errors are inevitable. The episode makes the case for incremental, streamed processing over batch jobs, along with exponential backoff and resumable job state.Validating output rigorously: Linting and automated tests are the floor, not the ceiling. Human review — designed specifically around AI-generated diffs — is the safeguard against subtle regressions in null handling, error logging, and load-bearing quirks the model can't know about.Knowing GPT-4-Turbo's blind spots: The model excels at mechanical, repetitive cleanup but has a tendency toward unnecessary abstraction and occasionally produces changes that are technically valid and practically baffling. Listeners get a realistic picture of both the wins and the failure modes.The throughline is that this approach works best when human judgment stays in the loop — using AI to automate drudgery while reserving architectural decisions for engineers who understand the codebase's history. If you enjoyed this episode, When Your AI Forgets the World Changed: Data Drift Detection Explained covers another critical dimension of building reliable AI systems in production. DEV

  2. 20小时前

    When Your AI Forgets the World Changed: Data Drift Detection Explained

    A machine learning model can be technically flawless the day it ships and still become a liability six months later — not because anyone broke it, but because the world it was trained on no longer exists. This episode of Development tackles one of the most underappreciated threats to production AI systems: data drift. Drawing on this in-depth guide to implementing online monitoring pipelines for AI systems, the episode walks through why drift happens, how to recognize its different forms, and what a practical, engineering-first response actually looks like. Here's what the episode covers: What data drift really is — the growing gap between what a model was trained to expect and what it's actually seeing in production, illustrated with examples from spam filtering, retail recommendations, and manufacturing quality control.Three distinct types of drift — concept drift (the relationship between inputs and outputs changes), covariate drift (the distribution of input data shifts), and label drift (the definition of the target itself evolves) — and why each demands a different response.Building a monitoring pipeline from the ground up — capturing live inputs and predictions in near real time using tools like Apache Kafka or AWS Kinesis, and establishing a historical baseline to compare against.Statistical methods for detecting drift — including the Kolmogorov-Smirnov test, ML-based drift detectors, and simpler approaches like tracking prediction-vs-outcome match rates over time.Automated alerting and response playbooks — why teams that set up drift metrics but skip the alert layer are still flying blind, and how to define a clear protocol before an alarm ever sounds.Two common pitfalls — alert fatigue from over-sensitive thresholds, and the mistake of ignoring external signals (regulatory changes, cultural shifts, supply chain events) that can predict drift before the metrics catch up.The episode closes with a reminder that drift isn't evidence of a poorly built model — it's evidence that the world keeps moving. The teams that treat monitoring as a first-class engineering discipline are the ones whose AI investments stay reliable and trustworthy over time. For more from the show on the sharp edges of deploying AI in production, don't miss Why Deploying LLMs on Serverless Is a Beautiful Disaster. DEV

  3. 1天前

    Real-Time ML Inference: Wrangling Kafka and TensorFlow Serving

    Real-time machine learning inference has moved from a competitive advantage to a baseline expectation. This episode of Development tackles the architectural and operational realities of building a streaming inference pipeline — one that can ingest live events, score them against an ML model, and return predictions in milliseconds. The discussion is grounded in the deep-dive on streaming ML inference with Kafka and TensorFlow Serving and covers everything from initial design decisions to the production pain points that only reveal themselves under real traffic. Here's what this episode walks through: Why real-time inference matters: Use cases like fraud detection, content recommendation, and autonomous systems illustrate why latency measured in minutes — or even seconds — is simply no longer acceptable.Kafka as the data backbone: How Kafka's high-throughput, fault-tolerant, and replayable event streaming model keeps a continuous flow of fresh feature data moving to the model — no waiting for a batch to assemble.TensorFlow Serving in production: Why Google's model-serving system goes beyond simple hosting, offering version management, live rollouts, and both REST and gRPC interfaces for flexible integration.Kafka configuration trade-offs: Practical guidance on partition count, message retention, schema registries, and monitoring consumer lag — the decisions that trip up teams early and quietly.Scaling and optimizing TensorFlow Serving: When to scale horizontally via Kubernetes, when to optimize the model itself (quantization, distillation, request batching), and why GPU acceleration isn't always the right first move.Resilience and observability: Building error handling that prevents cascading failures — retry caps, circuit breakers, graceful degradation — alongside a monitoring stack (Prometheus and Grafana) that tracks the metrics that actually matter.The episode closes with an honest framing: combining Kafka and TensorFlow Serving is less a one-time technology choice and more an ongoing operational commitment. The teams that get the most out of this stack are the ones who invest in understanding it deeply, tune it deliberately, and treat observability as a first-class concern from day one. More from the show: if you enjoyed this episode, check out Why Deploying LLMs on Serverless Is a Beautiful Disaster for another candid look at the operational complexity hiding inside modern ML deployments. DEV

  4. 2天前

    Why Deploying LLMs on Serverless Is a Beautiful Disaster

    Serverless computing promises infinite scale, zero infrastructure management, and pay-per-use economics — so the idea of running a large language model on top of it is understandably tempting. But the gap between that pitch and production reality turns out to be enormous. This episode of Development unpacks the very real architectural friction explored in this deep-dive on deploying large language models in serverless environments, walking through the core challenges and the hybrid strategies that actually hold up at scale. The conversation covers a lot of ground for engineers weighing up this architectural choice: Cold starts as a dealbreaker: When a serverless function wakes from idle, loading a multi-gigabyte LLM before processing a single token introduces multi-second delays that make real-time applications essentially non-functional — and "warm instance" workarounds quietly abandon the serverless cost model entirely.The statelessness mismatch: Serverless is built around clean-slate, stateless invocations, while LLMs depend on persistent conversational context. Offloading state to external caches or databases solves this in theory, but adds latency, new failure modes, and significant debugging complexity.Hard compute ceilings: Platform limits like AWS Lambda's 10 GB RAM and 15-minute execution cap are nowhere near sufficient for frontier-scale models. Teams are forced to choose between aggressive quantization or offloading inference to GPU-backed services — either way, the pure serverless architecture doesn't survive contact with the model.Storage and loading overhead: With no persistent in-memory model between invocations, every cold start may require downloading gigabytes from object storage, while persistent file system alternatives add their own cost and complexity.Security and isolation concerns: Ephemeral function instances handling sensitive user data, pulling open-source dependencies at runtime, and processing prompts vulnerable to injection attacks create a security surface that demands far more discipline than serverless's simplicity tends to encourage.The cost trap: Per-invocation billing sounds economical until LLM request volumes scale up. Compute-heavy, slow-generating models can turn a seemingly lean architecture into a budget crisis — making the hybrid model not just pragmatic, but financially necessary.The episode lands on a clear verdict: the teams that succeed with AI infrastructure aren't the ones chasing architectural purity. A hybrid approach — serverless handling lightweight orchestration, routing, and preprocessing, while dedicated GPU-backed infrastructure takes on serious inference workloads — consistently outperforms attempts at going fully serverless with a large model. Before committing to any LLM deployment strategy, it's worth pressure-testing the cold start behaviour, the context storage plan, and the cost model at realistic request volumes. For more on the NLP skills that underpin effective language model pipelines, check out the Development episode on Custom Tokenization Pipelines: The NLP Skill You Can't Afford to Skip. DEV

  5. 3天前

    The Future of Coding Is No Coding at All — And Why Developers Need to Adapt Now

    The rise of no-code and low-code platforms isn't a passing trend — it's a structural change in how software gets built. This episode of Development draws on the article exploring why coders need to adapt to a no-code world to examine what this shift actually means for working developers: not the end of the profession, but a fundamental redefinition of where developer value lives. Hosts walk through the economic logic behind the no-code movement, the tools reshaping production workflows, and the specific skills that will separate indispensable developers from those who get left behind. Key topics include: Why the economics have already shifted — businesses pay for outcomes, not effort, and no-code platforms are changing what outcomes cost to deliver.The 80/20 reality of no-code capability — visual platforms can handle the majority of typical application needs, but custom logic, security architecture, and complex integrations still demand human expertise.Tools worth knowing now — Cursor's AI-assisted development environment and V0.dev's prompt-to-component workflow are highlighted as practical entry points for developers looking to expand their toolkit.The developer-as-strategist role — the roles shrinking are those built around generating boilerplate; the roles growing are those that combine technical intuition with product thinking and business fluency.Skills that remain non-negotiable — systems thinking, API and integration knowledge, and data security expertise don't disappear when drag-and-drop tools enter the picture.The mindset shift that matters most — moving from "how much code can I write?" to "what's the best way to solve this problem with the tools available?" is framed as the single most important career adaptation.The episode closes with a clear-eyed look at what hybrid development teams will look like — and why the developers who can move fluidly between visual builders and complex backend systems will be the ones shaping what gets built next. More from the show: if you're thinking about the technical skills that set developers apart, don't miss Custom Tokenization Pipelines: The NLP Skill You Can't Afford to Skip. DEV

  6. 5天前

    Custom Tokenization Pipelines: The NLP Skill You Can't Afford to Skip

    Tokenization is the step most NLP developers treat as an afterthought — until their model starts mangling stock tickers, shredding hashtags, and treating multi-word entities like random word salad. This episode of Development makes the case that tokenization is a foundational design decision, not a checkbox, drawing on this in-depth guide to building custom tokenization pipelines for NLP models. If your model's behavior has ever felt inexplicably broken despite clean-looking data, the tokenizer is almost certainly where the story starts. The episode walks through the full landscape of tokenization approaches and explains why knowing the trade-offs — not just the defaults — is what separates functional NLP projects from fragile ones. Here's what's covered: Why whitespace tokenization fails at scale — languages without clear word boundaries, contractions, punctuation, emojis, and multilingual text all expose its limits almost immediately.Character-level tokenization — eliminates unknown-word problems entirely but produces sequences so long they become computationally punishing for transformer architectures.Rule-based and regex tokenization — powerful for structured, predictable text and extensible via tools like spaCy, but every edge case demands a new rule, and the edge cases never stop arriving.Subword tokenization (BPE, WordPiece, Unigram) — the backbone of modern large language models, handling out-of-vocabulary terms gracefully by breaking unfamiliar words into recognizable, reusable units.The "IKEA furniture" problem with pre-trained tokenizers — general-purpose tokenizers work until they meet domain-specific text (finance, medicine, social media), at which point their assumptions become liabilities.Performance at scale — why tokenization becomes a pipeline bottleneck long before most developers expect it to, and how tools like Hugging Face's Rust-powered tokenizers library help close the gap between speed and accuracy.The episode closes with a candid take: building a custom tokenization pipeline is genuinely difficult, unglamorous work — but getting it right pays dividends across everything downstream, from training speed to real-world generalization. For more on using AI to reduce tedious manual work in development workflows, check out the Development episode Stop Writing API Docs by Hand — Let AI Do the First Draft. DEV

  7. 6天前

    Stop Writing API Docs by Hand — Let AI Do the First Draft

    Every developer knows the feeling: the code ships clean, the tests pass, and then the documentation tab sits open, untouched, for days. API docs have a way of drifting out of sync with the codebase almost immediately after they're written — and the real cost isn't inconvenience, it's the compounding miscommunication that erodes team trust over time. This episode of Development explores how large language models are changing that dynamic by handling the grunt work of the first draft, drawing on this in-depth look at automating API documentation with AI. Here's what the episode covers: Why manual docs fail predictably — not from carelessness, but because code and documentation live in separate places and move at different speeds, turning docs into archaeological artifacts.How LLMs read your code — using pattern-matching across naming conventions, function signatures, and inline comments to infer intent and generate structured descriptions at scale.The "junior dev first draft" mental model — AI output isn't perfect, but it's evaluated against the real alternative: no docs, outdated docs, or docs that took hours to write and were stale by Friday.Continuous documentation via CI/CD integration — triggering regeneration on every merge or push so that docs become a natural byproduct of development, not a painful batch project.Getting better results from any tool — the outsized impact of descriptive naming conventions, having a team style guide for review passes, and knowing which domain-specific logic still needs a human touch.Honest cost considerations — weighing subscription-based platforms against growing open-source options, and how to think about ROI when developer time is the real constraint.The core argument isn't that AI replaces developer judgment — it's that it moves the developer further up the loop, out of the tedious parts and into the decisions that actually require expertise. More from the show: if you're interested in building AI systems from the ground up, check out Training a Diffusion Model from Scratch: A Developer's Real Guide. DEV

  8. 7月22日

    Training a Diffusion Model from Scratch: A Developer's Real Guide

    Most developers who work with generative AI stop at the API layer — and that's fine, until curiosity kicks in. This episode of Development pulls back the curtain on what it genuinely takes to train a diffusion model from the ground up, drawing on the step-by-step guide to training a diffusion model for custom image generation published at DEV. Whether the goal is a specialized creative tool, a proprietary image pipeline, or simply a deeper understanding of how these systems work, this episode treats the topic with the seriousness it deserves — no hand-waving, no skipped steps. Here's what the episode covers: How diffusion models actually work — the intuition behind learning to reverse a noise process, and why that iterative denoising approach produces such high-quality outputs.Data curation as a first-class concern — why the visual distribution of your training set directly shapes what your model can and can't generate, and what "good enough" data actually looks like in practice.Hardware and environment setup — the real GPU requirements, cloud provider options, and why adding an experiment-tracking tool like Weights and Biases from day one saves significant pain later.Architecture choices and the U-Net backbone — what makes U-Nets well suited to denoising tasks, and why starting from an existing open-source implementation beats building from absolute zero.Reading the training signals — how to interpret loss curves and early sample outputs, what normal early-stage fuzziness looks like versus a run that's genuinely broken, and how to troubleshoot common failure modes like overfitting, underfitting, and out-of-memory errors.Fine-tuning and deployment — how a shorter, focused training pass can specialize a general base model, and practical ways to wrap a finished model in a REST API, a local tool, or an interactive dashboard.The honest takeaway from this episode: training a diffusion model from scratch demands compute, patience, and careful iteration — but the reward isn't just a working model. It's a mechanistic understanding of generative AI that holds its value long after the surface-level tooling has moved on. For more on making deep learning models leaner without sacrificing what matters, check out the earlier episode Neural Network Quantization: Shrinking Models Without Losing Accuracy. DEV

查看全部 56 集

关于

Software and AI development podcast. We cover all things software development, including today's advanced AI development tricks and techniques.

信息

  • 创作者
    Eric Lamanna
  • 活跃年份
    2026年
  • 单集
    56
  • 分级
    儿童适宜
  • 版权
    © 2026 DEV.co
  • 节目网站
    DEV