#238 - AI is Smart Until It's Dumb: Why LLM Will Fail When You Least Expect It - Emmanuel Maggiori

Why does an AI that brilliantly generates code suddenly fail at basic math? The answer explains why your LLM will fail when you least expect it.

In this episode, Emmanuel Maggiori, author of “Smart Until It’s Dumb” and “The AI Pocket Book,” cuts through the AI hype to reveal what LLMs actually do and, more importantly, what they can’t. Drawing from his experience building AI systems and witnessing multiple AI booms and busts, Emmanuel explains why machine learning works brilliantly until it makes mistakes no human would ever make.

He shares why businesses repeatedly fail at AI adoption, how hallucinations are baked into the technology, and what developers need to know about building reliable AI products.

Whether you’re implementing AI at work or concerned about your career, this conversation offers a grounded perspective on navigating the current AI wave without getting swept away by unrealistic promises.

Key topics discussed:

• Why AI projects fail the same way repeatedly
• How LLMs work and why they brilliantly fail
• Why hallucinations can’t be fixed with better prompts
• Why self-driving cars still need human operators
• Adopting AI without falling into hype traps
• How engineers stay relevant in the AI era
• Why AGI predictions are mostly marketing
• Building valuable products in boring industries

Timestamps:

• (00:00:00) Trailer & Intro
• (00:02:32) Career Turning Points
• (00:06:41) Writing “Smart Until It’s Dumb” and “The AI Pocket Book”
• (00:08:14) The History of AI Booms & Winters
• (00:11:34) Why Generative AI Hype is Different Than the Past AI Waves
• (00:13:26) AI is Smart Until It’s Dumb
• (00:16:45) How LLM and Generative AI Actually Work
• (00:22:53) What Makes LLMs Smart
• (00:27:25) Foundational Model
• (00:30:01) RAG and Agentic AI
• (00:34:09) Tips on How to Adopt AI Within Companies
• (00:37:56) How to Reduce & Avoid AI Hallucination Problem
• (00:45:49) The Important Role of Benchmarks When Building AI Products
• (00:50:57) Advice for Software Engineers to Deal With AI Concerns
• (00:56:49) Advice for Junior Developers
• (00:59:34) Vibe Coders and Prompt Engineers: New Jobs or Just Hype?
• (01:01:55) The AGI Possibility
• (01:07:23) Three Tech Lead Wisdom

_____

Emmanuel Maggiori’s Bio
Emmanuel Maggiori, PhD, is a software engineer and 10-year AI industry insider. He has developed AI for a variety of applications, from processing satellite images to packaging deals for holiday travelers. He is the author of the books Smart Until It’s Dumb, Siliconned, and The AI Pocket Book.

Follow Emmanuel:

• LinkedIn – linkedin.com/in/emaggiori
• Website – emaggiori.com

Like this episode?
Show notes & transcript: techleadjournal.dev/episodes/238.
Follow @techleadjournal on LinkedIn, Twitter, and Instagram.
Buy me a coffee or become a patron.