Automatic

Eric Lamanna

Podcast for Automatic.co and LLM.co, the AI automation specialists.

  1. -6 h

    AI Latency Budgets: Why Seconds Kill Products

    Speed is a trust signal, not just a technical metric — and in AI products, it may be the difference between a loyal user and a silent churn. This episode of Automatic draws on the original article on AI latency budgets and why seconds kill products to make the case that every team shipping an AI feature needs to treat response time as a first-class design constraint, not an afterthought. Unlike a traditional web request, an AI pipeline chains together a dozen moving parts — intent detection, retrieval, reranking, prompt construction, model inference, post-processing, and client rendering — each silently consuming milliseconds. Without a formal budget, those costs compound invisibly. The episode walks through what it actually takes to define, instrument, and enforce a latency budget across a modern AI stack, covering: The human thresholds that define "fast enough" — why sub-second interactions feel instantaneous, why three seconds marks the edge of trust, and how the same answer delivered faster is perceived as smarter and safer.How to structure a component-level budget — assigning explicit millisecond allowances to each pipeline stage (input sanitization, retrieval, reranking, generation, streaming) so tradeoffs become visible and negotiable instead of political.Why server-side averages are a trap — the case for client-side instrumentation, tail latency percentiles (P95 and P99), correlation IDs, and structured spans that map the full journey from tap to paint.Streaming and progressive disclosure as perception tools — how showing partial output within a few hundred milliseconds bends the user's patience curve without changing total compute time.The overlooked latency culprits — tokenization overhead, vector store recall budgets, cold starts, payload size, TLS negotiation, and geographic distance between compute and user all adding up faster than model inference alone.Speed as an organizational culture, not just an engineering target — why teams that celebrate latency wins ship faster products, and how performance gates in the build pipeline prevent regressions from sneaking into production.The episode closes with a practical playbook: name an end-to-end target, assign component allowances, parallelize aggressively, right-size models for each pipeline stage, and instrument everything. If you enjoyed this one, don't miss How Private LLMs Reduce Operational Risk for Finance Teams for another look at building AI systems that earn — and hold — user trust. Automatic

  2. -16 h

    Bringing Agentic AI In-House: Private LLMs That Act, Not Just Chat

    Most AI deployments inside companies stop at conversation — ask a question, get an answer, then go do the work yourself. This episode of Automatic examines the next leap: agentic AI systems that don't just respond but genuinely act, and why a growing number of organizations are choosing to run them entirely on private infrastructure. Drawing on this deep-dive on private agentic LLMs, the episode walks through what it actually takes to move from a chat assistant to an autonomous co-worker operating inside your own walls. Here's what the episode covers: The agentic shift explained: How modern LLMs move beyond text generation to execute multi-step tasks — querying data warehouses, filing tickets, routing approvals, and sending outputs — without human hand-offs at every stage.Three enabling technologies: Toolformer-style API-aware training, long-context architectures that preserve a plan across many steps, and fine-grained policy engines that block unsafe actions before they fire.Why private deployment wins: The compounding case for keeping agentic AI behind your own firewall — compliance obligations (HIPAA, SOC 2, GDPR), competitive sensitivity of the data in real prompts, direct control over guardrails, and more predictable infrastructure costs versus token-based public APIs.A five-layer architecture: The episode unpacks the distinct roles of the core inference layer, a vector-database memory and planning layer, a tooling and orchestration layer (LangChain or custom schemas), a policy enforcement sandbox, and a human-in-the-loop approval portal — and explains why clean boundaries between layers make the whole system maintainable.Governance that holds up in practice: Pre-deployment alignment through curated training data and scoped permissions, plus real-time oversight via exhaustive logging, rate limits, mandatory approvals, and "shadow mode" pilots where agents suggest but cannot act until accuracy benchmarks are met.A phased rollout playbook: Start narrow — legal-hold reminders, CRM cleansing, nightly ops reports — instrument everything, then widen autonomy incrementally as the system earns organizational trust and each small win funds the next infrastructure step.The episode closes with the strategic argument that organizations crossing this threshold now — with the right layered architecture and rigorous governance — will set a meaningfully higher bar for what intelligent automation looks like. More from the show: if you want to see how a specific regulated industry is already navigating this transition, check out the episode From Paper Stacks to Private LLMs: How Insurers Are Reinventing Claims. LLM

  3. -16 h

    Prompt Caching: How to Make LLMs Faster and Cheaper Without Cutting Corners

    Every time an LLM application sends the same background context — system prompts, schema definitions, policy documents — without caching, it pays for that context all over again. At scale, across thousands of daily requests, that overhead compounds fast. This episode of Automatic unpacks the full guide to prompt caching for LLM applications, exploring how teams can dramatically reduce token spend and response latency while keeping output quality high — or even improving it. The episode covers the core mechanics of prompt caching and works through the practical architecture decisions, common failure modes, and design patterns that separate a well-built caching layer from one that quietly causes problems: What caching actually preserves — not full responses, but reusable intermediate artifacts like retrieval plans, chunk summaries, tool-call templates, and formatting scaffolds.Why quality can improve — caching deterministic steps removes variance from parts of a pipeline that should behave consistently, letting the model focus fresh generation where it actually matters.Where caching fits in real stacks — retrieval and grounding systems, tool-use orchestration, and structured generation each have distinct cacheable units and different risk profiles.Key design decisions — how to define the right cacheable unit, build cache keys that balance hit rate against staleness risk, and set expiration and invalidation policies that won't surprise the team during a model upgrade.Practical patterns that pay off quickly — intent normalization before routing, plan-then-execute splits that cache blueprints, and templated outputs with live-filled slots.The traps to avoid — over-caching personal or rapidly shifting context, hidden coupling between pipeline layers, and applying reuse in domains like legal or medical guidance where even structural similarity can mislead.The episode also addresses safety and trust: why cache hits should still pass through lightweight validators, how to log cache provenance for observability, and what privacy hygiene looks like when personal data and secrets are in play. The closing argument is that prompt caching, done with discipline, isn't an optimization hack — it's closer to craftsmanship, and the underlying pattern will remain valuable even as context windows and model capabilities continue to expand. For more on how AI is reshaping complex, document-heavy industries, listen to From Paper Stacks to Private LLMs: How Insurers Are Reinventing Claims. Automatic

  4. -2 j

    From Paper Stacks to Private LLMs: How Insurers Are Reinventing Claims

    Insurance claims have never been clean, simple, or fast — and for decades, the industry has relied on rules-based software and armies of human reviewers to manage the mess. This episode of Automatic examines how that model is changing, drawing on this deep dive into private LLMs and claims data parsing to explore why large language models are proving to be one of the most consequential tools insurers have ever adopted. The shift goes well beyond efficiency — it touches auditability, regulatory compliance, and the day-to-day experience of policyholders navigating some of the hardest moments of their lives. The episode walks through the full arc of how private LLMs are being deployed inside insurance operations, from raw document ingestion to final adjuster review. Key topics include: Why claims data resists automation: A single claim can contain narrative reports, scanned PDFs, billing codes, email threads, and legacy attachments — a mix that traditional rules-based systems were never built to handle.Context-aware reading at scale: Unlike keyword-matching software, LLMs understand that identical language carries different meaning depending on where it appears in a document — a distinction that previously required careful human reading.The ingestion and extraction pipeline: How optical character recognition, layout analysis, and entity extraction work together to turn unstructured claims documents into structured, cited, auditable outputs.Policy reasoning and conflict detection: How well-designed systems compare claim details against live policy language — flagging potential exclusions, recommending reserve amounts, and explaining their reasoning in plain language for adjuster review.Privacy and governance as non-negotiables: Why insurers deploying these tools keep everything inside controlled networks, with locked-down data residency, access logging, encryption, and regression testing built into the model update cycle.Retrieval-augmented generation as a trust layer: How grounding model outputs in retrieved, up-to-date sources — rather than relying on training data alone — addresses the hallucination problem that makes enterprise AI adoption so risky in regulated industries.The episode also addresses the human side of implementation: how LLMs are being positioned as companion services alongside legacy claims platforms, why adjusters remain essential for judgment-heavy decisions, and what concrete success metrics — handling time, straight-through processing rates, claim reopening rates — actually reveal about whether a deployment is working. For more on how this technology is reshaping regulated industries, the episode How Private LLMs Reduce Operational Risk for Finance Teams covers closely related ground from a financial-services perspective. LLM

  5. -4 j

    How Private LLMs Reduce Operational Risk for Finance Teams

    Operational risk in finance rarely announces itself with a dramatic failure — it hides in a four-cent ledger discrepancy, a Friday afternoon rubber-stamp, or a compliance policy sitting unread in a shared folder. This episode of Automatic breaks down how private large language models are giving finance teams a fundamentally different kind of control, drawing on this in-depth look at how private LLMs lower operational risk for finance teams. Rather than bolting more rules onto aging infrastructure, private LLMs work with the messy, unstructured reality of modern finance — and they do it entirely inside the firm's own walls. The episode covers the full arc of where and how this architecture makes a difference: The hidden shape of operational risk: Why most financial errors are quiet, contextual, and invisible to traditional rules-based controls — and why that gap has been widening as data environments grow more complex.What "private" actually means: A private LLM lives inside the firm's own infrastructure, keeping every query, output, and data reference behind the organization's encryption perimeter — a hard requirement under strict data residency rules, not just a preference.Faster, cleaner reconciliations: How private models cross-reference multi-system ledgers at speed, surface variances, and draft explanations for human review — shrinking the month-end close from an all-nighter to a routine afternoon task.Real-time compliance enforcement: Embedding current policies directly into the model means every request is scanned against live rules before anything moves forward, with audit evidence tagged and organized as it accumulates rather than assembled under pressure.Democratizing analytical access: Breaking the bottleneck that forces non-technical staff to queue behind specialists for risk and exposure data — so decision-makers get answers in plain English, in seconds, when it matters.A compounding feedback loop: Each human accept-or-reject interaction becomes a training signal, quietly improving the model's fit to a specific desk and workflow over time — without manual retraining cycles.The episode also examines how reducing third-party integrations trims the vendor attack surface, how logged exceptions and timed reconciliation tasks feed measurable key risk indicators, and why this shift represents a strategic edge rather than an incremental upgrade. For more on how machine learning models handle imperfect data, check out the earlier episode Synthetic Labels: Training Machine Learning Models Without the Truth. LLM

  6. -5 j

    Synthetic Labels: Training Machine Learning Models Without the Truth

    Ground truth is expensive, slow, and often the single biggest bottleneck between a machine learning idea and a working model. This episode of Automatic explores a growing alternative: synthetic labels — algorithmically generated targets that stand in for human annotation, sometimes dramatically compressing the time and cost of getting a model into production. The approach is detailed in the source article on training without the truth, and this episode walks through the key ideas with enough depth to be immediately useful. The episode covers the three main flavors of synthetic labeling and the discipline required to make any of them work reliably: Pseudo-labeling — training a model on a small seed of human-labeled data, then using its highest-confidence predictions to expand the training set into a much larger unlabeled pool.Weak supervision — combining noisy programmatic labeling functions (rules, heuristics, keyword lists) through a label model that produces probabilistic targets a downstream classifier can learn from.Knowledge distillation — using a large teacher model's soft probability outputs as labels for a smaller student, preserving nuanced inter-class information that hard labels discard.The generator–filter–learner loop — the production architecture connecting label proposal, curation (confidence thresholds, diversity sampling, deduplication), and model training into an iterative cycle rather than a one-shot pipeline.Calibration and bias management — why overconfident generators bully student models into confident mistakes, and how techniques like temperature scaling and label smoothing reduce that risk; plus why evaluation slices across demographic and geographic groups are non-negotiable.When synthetic labels are and aren't appropriate — the episode is direct about the limits: moderate-stakes routing tasks are a strong fit; high-stakes clinical or legal decisions should lean hard on verified ground truth.A recurring theme is the "sacred holdout set" — a small, clean, human-verified test set that never touches training and gates every deployment decision. The episode also touches on how synthetic labeling pairs naturally with self-supervised learning and active learning, forming a practical toolkit for teams with more ambition than annotation budget. For more from the show, the episode Waking the Sleeping Giant: How LLMs Turn Legacy Databases Into Allies explores another angle on making AI work with imperfect, real-world data conditions. Automatic

  7. -6 j

    Waking the Sleeping Giant: How LLMs Turn Legacy Databases Into Allies

    For most organizations, legacy databases are a paradox: they contain some of the most valuable institutional knowledge in the business, yet extracting anything useful from them demands specialized skills, undocumented workarounds, and a tolerance for friction that most modern teams simply don't have. This episode of Automatic digs into how that dynamic is shifting, drawing on this deep-dive on turning legacy databases into intelligent assistants — and what it means for the enterprises that have been quietly sitting on goldmines of inaccessible data. The episode walks through the full arc of what it takes to place a private large language model between business users and aging database infrastructure — from the unglamorous groundwork to the surprisingly far-reaching operational payoffs. Key points covered include: Why legacy databases resist access: Decades of accumulated complexity — cryptic field names, undocumented stored procedures, departed subject-matter experts, and brittle interdependencies — make even routine queries a high-stakes exercise.The conversational layer explained: A private LLM acts as a semantic bridge, translating plain-English questions into precise, optimized SQL without requiring the user to know table structures, join logic, or fiscal-quarter definitions.Schema preparation as a prerequisite: Before any assistant can be useful, teams must surface hidden assumptions, trace undocumented relationships, and formalize business definitions — work that pays dividends far beyond the AI implementation itself.Governance baked in from the start: Sensitive fields, access controls, output watermarking, and real-time policy enforcement aren't optional add-ons — they're what separates a genuinely useful deployment from a compliance liability.From reactive to proactive intelligence: A mature implementation doesn't just answer questions — it surfaces anomalies, flags performance patterns, and acts as an advisor with perfect recall and no political agenda.Measuring what matters: The episode emphasizes setting concrete success metrics before go-live — query turnaround time, manual exports eliminated, reconciliation hours saved — and connecting those numbers to leadership-level outcomes.The throughline is a reframe that's both practical and consequential: the problem was never the data itself, but the assumption that accessing it had to be painful. For teams curious about the architectural thinking behind real-time data pipelines, Message Brokers: Who's Actually in Charge Here? is a natural companion listen. More from the show can be found there. LLM

  8. 21 juil.

    Message Brokers: Who's Actually in Charge Here?

    Messaging systems are everywhere in modern software, yet one fundamental question rarely gets a straight answer: does control live in the broker, or in the services that use it? This episode of Automatic works through the architecture of message brokers — exploring how responsibility is actually divided, where things go wrong, and what well-designed systems get right. It draws on the source article on message broker control vs. coordination to give teams a sharper mental model for building with brokers intentionally. Here's what the episode covers: Control vs. coordination: The broker owns how messages move; services own what those messages mean — and confusing the two is where architectures quietly unravel.Edges matter most: Producers shape message frequency and format; consumers govern back pressure and idempotency. The broker can surface symptoms of poor judgment at the edges, but it can't fix them.The temptation of broker features: When teams push business logic into routing rules or stateful broker filters, what feels like a shortcut becomes a maintenance maze — keeping the broker focused on transport protects long-term clarity.Message contracts as the real lever: A stable, well-versioned schema is more powerful than any broker capability. Additive changes, schema registries, and disciplined review processes reduce the broker to a fast, reliable pipe.Delivery guarantees and ordering myths: At-least-once delivery with idempotent consumers is the most practical foundation; chasing global ordering or exactly-once semantics often shifts hidden complexity rather than eliminating it.Governance, observability, and ownership: Topic naming conventions, lag dashboards, correlation identifiers, and clear stewardship aren't bureaucracy — they're the feedback loops that make control feel like confidence rather than guesswork.The episode closes with a straightforward verdict: messaging systems run on shared custody. Brokers set the rails; services decide where to go and what to carry. When that boundary is clear — backed by strong contracts, idempotent consumers, and honest observability — teams ship faster and sleep better. For more from the show, check out the episode Why Autonomous AI Agents Need On-Prem Isolation, which explores another dimension of infrastructure control in automated systems. Automatic

Tout afficher (58)

À propos

Podcast for Automatic.co and LLM.co, the AI automation specialists.

Informations

  • Création
    Eric Lamanna
  • Années d’activité
    2026
  • Épisodes
    58
  • Classification
    Tous publics
  • Copyright
    © 2026 Automatic.co
  • Site web de l’émission
    Automatic