Send us Fan Mail 📖 Read: https://helioxpodcast.substack.com/publish/post/218050132 Why teaching neural networks to think in curved, folded space — not flat grids — could cut AI's power bill, stop it from hallucinating, and finally give humans a steering wheel for the black box. Your AI can write code and pass the bar exam — so why does it also hallucinate, drain a small country's worth of electricity, and turn dead in a huge fraction of its own neurons? This episode traces the answer to a surprisingly simple culprit: for a decade, AI has been forced to process the world using flat, grid-based math that was never built for a curved, rotating, three-dimensional reality. We trace the escape route — a shift toward curved geometry called the Grassmann manifold — through the real engineering that's made it possible, including the GRNet architecture built at ETH Zurich, and follow it all the way to a strange, well-documented phenomenon called grokking, where a stuck model suddenly, physically "clicks" into understanding. Along the way: why models hallucinate, why some neurons quietly die, how researchers turned an O(L²) bottleneck into a linear one, and how this same curved math is starting to hand humans literal sliders to steer AI-generated faces and bodies. Evidence-based, deeply researched, and gently skeptical throughout — this is Heliox: Where Evidence Meets Empathy. References: Building Deep Networks on Grassmann Manifolds and twelve other papers Chapters 00:00 Cold Open: The Shadow on the Wall 02:14 A New Paradigm for AI 04:07 How Euclidean Neural Networks Work 07:18 The Quadratic Scaling Problem 09:33 The Memory Wall 11:00 Dying ReLUs 14:28 Escape Route: The Grassmann Manifold 18:30 Neural Superposition Explained 21:41 The k n/2 Threshold 23:36 Inside GPT-2's Geometry 29:10 Engineering GRNet 32:22 GDLNet and Linear Scaling 38:05 Grokking: The AI's Aha Moment 45:17 Building a Steering Wheel for AI 53:46 Closing Thought: Is Your Brain a Manifold? This is Heliox: Where Evidence Meets Empathy Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter. Breathe Easy, we go deep and lightly surface the big ideas. Support the show Disclosure: This podcast uses AI-generated synthetic voices for a material portion of the audio content, in line with Apple Podcasts guidelines. We make rigorous science accessible, accurate, and unforgettable. Produced by Michelle Bruecker and Scott Bleackley, it features reviews of emerging research and ideas from leading thinkers, curated under our creative direction with AI assistance for voice, imagery, and composition. Systemic voices and illustrative images of people are representative tools, not depictions of specific individuals. We dive deep into peer-reviewed research, pre-prints, and major scientific works—then bring them to life through the stories of the researchers themselves. Complex ideas become clear. Obscure discoveries become conversation starters. And you walk away understanding not just what scientists discovered, but why it matters and how they got there. Independent, moderated, timely, deep, gentle, clinical, global, and community conversations about things that matter. Breathe Easy, we go deep and lightly surface the big ideas. Spoken word, short and sweet, with rhythm and a catchy beat. http://tinyurl.com/stonefolksongs
