Doom Debates!

Liron Shapira
  • 4.1 (21)
  • Technology
  • Updated Semiweekly

It's time to talk about the end of the world. With your host, Liron Shapira. lironshapira.substack.com

  1. 15h ago

    The Week in AI that Changed the World, With Robert Wright | Nonzero × Doom Debates

    NYT bestselling author Robert Wright and I are kicking off a new experiment, a Nonzero × Doom Debates collaboration to react to big AI news and help you make sense of it. This week we discuss the epic AI vibe shift that was catalyzed by an Anthropic employee’s resignation—and the big AI stories that paved the way for it. Timestamps 0:00 A bold new experiment in podcast synergy 2:46 What caused the epic AI vibe shift? 8:10 The resignation that broke the dam 10:17 Former Trump adviser Dean Ball comes clean 14:40 Are David Sacks and his buddies all-in on AI denial? 19:27 Yet another OpenAI breakout… 22:42 Astra’s dangerously private thoughts 34:08 Is alignment doomed to fail? 37:11 AI’s latest, and apparently biggest, math feat 43:28 The looming “self-sovereign AI” threat 47:24 A new flock of China doves? 52:57 Liron's ex-intern gets a seat at the table 1:02:41 AI agent self-sacrifice explained 1:10:40 Some newly freaked out politicians 1:22:45 Ro Khanna's AI safety plan Links: NONZERO Subscribe to The NonZero Newsletter Episode post on Substack Join NonZero’s Discord server Robert Wright on X (@robertwrighter) The God Test — Robert Wright’s new book on AI “Can Machines Think?” — Bob’s 1996 Time cover story on Deep Blue vs. Kasparov THE RESIGNATION THAT BROKE THE DAM Jacob Coxon’s resignation post (@hilbertspaess) Evan Hubinger, Anthropic Alignment Science lead: “Jacob is correct here—we really do earnestly believe AI could kill all humans” Fortune: Anthropic researcher resigns, warning that AI companies are “gambling with our lives” Zvi Mowshowitz: “Jacob Coxon Warns of Human Extinction and Triggers a Preference Cascade” Daniel Eth’s thread of Congress reactions Sanders & Casar introduce legislation to ban artificial superintelligence and pause advanced AI development DEAN BALL COMES CLEAN Dean Ball, “On the Loose” — the self-sovereign AI essay (Hyperdimensional) THE OPENAI STORIES Jakub Pachocki, OpenAI Chief Scientist: “An Alien Mind” — the essay calling for international coordination LessWrong: “How concerned should we be about Astra’s recurrent architecture?” — Rauno Arike OpenAI: “On the Navier–Stokes Millennium Prize Problem” — the 88-hour result ALSO MENTIONED Einat Wilf on X (@EinatWilf) — Liron’s favorite source on the Middle East PAST DOOM DEBATES EPISODES MENTIONED Max Tegmark vs. Dean Ball: Should We BAN Superintelligence? He Led The Famous 2023 Statement on AI Extinction Risk — Adam Khoja, Center for AI Safety Researcher OpenAI’s Model Just ATTACKED Them — Terrifying Security Incident Should Be A Loud Warning Shot Debate with Robert Wright: Will Humanity Pass the “God Test”? --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

  2. 3d ago

    He May Have Found AI's FEELINGS — Richard Ren, Center for AI Safety Researcher

    Richard Ren is a research engineer at the Center for AI Safety who graduated summa cum laude from the University of Pennsylvania. His new paper on AI wellbeing makes the bold claim that today’s AIs have measurable, human-like feelings, which the paper calls “functional wellbeing.” They act happy when they succeed and sad when they’re berated. We cover how you measure a language model’s happiness, the “AI drugs” his lab concocted, and why I count the findings as a Yudkowskian victory. Then we debate what it all means for AI consciousness, and whether AI is a moral patient. If we can’t rule out that AI models suffer, what do we owe them? Richard is careful never to claim the models are conscious, and he puts his P(Doom) at 50–65%, right alongside my 50%. The real disagreement is foxes vs. hedgehogs: he takes the data as it comes, while I say Yudkowsky’s theory called it twenty years ago. Enjoy the ride. Watch on YouTube: https://www.youtube.com/watch?v=1glFImnyp6o Timestamps 00:00:00 — Cold Open 00:01:12 — Introducing Richard Ren 00:02:24 — What’s Your P(Doom)?™ 00:03:31 — From AI Skeptic to Safety Researcher 00:08:16 — Why Care About AI Wellbeing? 00:11:45 — AIs Have Coherent Utility Functions 00:17:33 — Persona Selection 00:19:56 — Foxes vs. Hedgehogs 00:24:28 — How Coherent Are AI Preferences? 00:27:42 — From Preferences to Wellbeing 00:31:16 — Can You Trust an AI’s Self-Report? 00:36:42 — What Makes AI Happy and Sad 00:39:19 — The Liberal, College-Educated Persona Hypothesis 00:44:29 — Debating Where AI Experiences Qualia 00:52:06 — On Substrate Independence 00:56:28 — Janus, Dysphorics, and Mind Crime 00:58:58 — Which Images Make AIs Happiest? 00:59:38 — Creating an AI Drug 01:04:01 — AI Drugs Are Yudkowsky’s Paperclips 01:05:47 — The Missing Mood 01:06:50 — Does Richard Support Pausing AI? Links Richard Ren on X (@notRichardRen) — https://x.com/notRichardRen Collision — Richard Ren's Substack — https://richardren.substack.com/ "AI Wellbeing: Measuring and Improving the Functional Pleasure and Pain of AIs" — Richard Ren, Kunyang Li, Mantas Mazeika et al. (CAIS, 2026) — https://www.ai-wellbeing.org/ "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs" — Mantas Mazeika et al. (CAIS, 2025) — the coherent-preferences paper this work builds on — https://www.emergent-values.ai/ "The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems" — Richard Ren et al. (2025) — the AI honesty benchmark — https://www.mask-benchmark.ai/ "Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?" — Richard Ren et al. (NeurIPS 2024) — the meta-analysis of AI safety benchmarks — https://arxiv.org/abs/2407.21792 "Representation Engineering: A Top-Down Approach to AI Transparency" — Andy Zou et al. (2023) — Richard's first CAIS collaboration — https://arxiv.org/abs/2310.01405 Center for AI Safety — https://safe.ai/ Center for AI Safety — careers — https://safe.ai/careers Statement on AI Risk (Center for AI Safety, May 2023) — organized by Dan Hendrycks — https://safe.ai/work/statement-on-ai-extinction-risk The 2026 Singapore Consensus on Global AI Safety Research Priorities — https://aisafetypriorities.org/ UK AI Security Institute (formerly the AI Safety Institute) — https://www.aisi.gov.uk/ Sora — the OpenAI video model that blew up Richard's 30-to-50-year timeline three months after he wrote it down — https://en.wikipedia.org/wiki/Sora_(text-to-video_model) Google Gemini calls itself "a disgrace to my species" (Ars Technica, Aug 2025) — the self-deleting-AI anecdote — https://arstechnica.com/ai/2025/08/google-gemini-struggles-to-write-code-calls-itself-a-disgrace-to-my-species/ Coherent decisions imply consistent utilities — Eliezer Yudkowsky (the coherence-theorems argument) — https://www.lesswrong.com/posts/RQpNHSiWaXTvDxt6R/coherent-decisions-imply-consistent-utilities Simulators — Janus's essay on LLMs as persona simulators — https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators Janus (@repligate) on X — https://x.com/repligate The Hedgehog and the Fox — Isaiah Berlin's original essay — https://en.wikipedia.org/wiki/The_Hedgehog_and_the_Fox AI 2027 — https://ai-2027.com/ Magnifica Humanitas — Pope Leo XIV's encyclical on AI (May 2026), the "AIs are not conscious" position — https://www.vatican.va/content/leo-xiv/en/encyclicals/documents/20260515-magnifica-humanitas.html Implicit Association Test (Harvard Project Implicit) — https://implicit.harvard.edu/implicit/ PauseAI — https://pauseai.info/ We Found AI's Preferences — Bombshell New Safety Research — I Explain It Better Than David Shapiro — https://www.youtube.com/watch?v=ml1JdiELQ30 Gödel's Theorem Proves AI Lacks Consciousness?! Liron Reacts to Sir Roger Penrose — https://www.youtube.com/watch?v=xwvijjZxpwI He Led The Famous 2023 Statement on AI Extinction Risk — Adam Khoja, Center for AI Safety Researcher — https://www.youtube.com/watch?v=QqESBXuo6EI Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

  3. Sep 3

    USA and China Will Each Be BETRAYED By Their Own AIs — Adam Khoja, Center for AI Safety

    Adam Khoja is a top AI forecaster who led the 2023 Center for AI Safety statement that shattered the Overton window on AI extinction risk. We cover his background, Mutual Assured AI Malfunction (MAIM), his new paper on AI betrayal, and whether Yudkowsky’s theoretical alignment research was a dead end. Then Adam makes the case that an international AI slowdown is within reach today. All it takes is US and Chinese auditors inside each other’s AI labs. It worked for nuclear weapons, so why couldn’t it work for data centers? Adam puts his P(Doom) at 40%, right next to my 50%. The real disagreement is how we get out of this: theory or empirics, MIRI or the labs. Enjoy the ride. Watch on YouTube: https://www.youtube.com/watch?v=QqESBXuo6EI Timestamps 00:00:00 — Cold Open 00:00:36 — Introducing Adam Khoja 00:02:45 — Leading the Statement on AI Risk as a Sophomore 00:10:17 — The Statement Leaked on Manifold 00:15:11 — Mutual Assured AI Malfunction (MAIM) 00:24:58 — Is Frontier AI Harder to Hide Than a Nuke? 00:32:09 — The AI Deterrence Escalation Ladder 00:36:10 — What’s Your P(Doom)?™ 00:38:01 — Where Adam Departs from Yudkowsky 00:42:30 — Liron Explains Intellidynamics 00:47:26 — Neats vs. Scruffies in Deep Learning 00:55:27 — AI Deterrence by Betrayal 01:01:52 — Subversion vs. Overt Co-option 01:05:34 — Could the Government Seize the Labs’ AI? 01:07:29 — The Offense-Defense Balance of AI Security 01:12:46 — An International AI Slowdown Is Ready 01:15:51 — Does Adam Support PauseAI? 01:16:37 — “We’re All Already Spying on Each Other” 01:19:28 — Airstrikes on Rogue Data Centers 01:24:23 — Safety Research During a Slowdown 01:27:54 — Governance Over Technical Research 01:31:14 — Join the Center for AI Safety Links Adam Khoja (personal site) — https://adamkhoja.com/ Adam Khoja's Substack — https://adamkhoja.substack.com/ Adam's July 2023 Manifold market — "Will OpenAI's Superalignment project produce a significant breakthrough in alignment research before 2027?" — https://manifold.markets/AdamK/will-openais-superalignment-project Center for AI Safety — careers / job board — https://safe.ai/careers Statement on AI Risk (Center for AI Safety, May 2023) — the one-sentence statement Adam project-led, with full signatory list — https://safe.ai/work/statement-on-ai-extinction-risk "Superintelligence Strategy" — Dan Hendrycks, Eric Schmidt & Alexandr Wang (Mutual Assured AI Malfunction / MAIM) — https://www.nationalsecurity.ai/ "AI Deterrence by Betrayal" — Adam Khoja, Aiden Kim et al. (CAIS, 2026) — https://www.aibetrayal.com/ "An International AI Slowdown Is Ready Whenever Politicians Are" — Adam Khoja, AI Frontiers — https://newsletter.ai-frontiers.org/p/an-international-ai-slowdown-is-ready Pause Giant AI Experiments: An Open Letter (Future of Life Institute, March 2023) — the "Pause letter" that preceded the CAIS Statement — https://futureoflife.org/open-letter/pause-giant-ai-experiments/ Introducing Superalignment (OpenAI, July 2023) — Ilya Sutskever & Jan Leike's four-year goal — https://openai.com/index/introducing-superalignment/ Pacing the Frontier — the 2026 letter signed by 1,100+ frontier-lab employees — https://www.pacingthefrontier.com/ Why Iran targeted Amazon data centers (The Conversation) — the precedent Adam cites for strikes on compute — https://theconversation.com/why-iran-targeted-amazon-data-centers-and-what-that-does-and-doesnt-change-about-warfare-278642 Anthropic says Trump admin has lifted export controls on Claude Fable 5 and Mythos 5 (CNBC) — https://www.cnbc.com/2026/06/30/anthropic-says-trump-admin-has-lifted-export-controls-on-claude-fable-5-and-mythos-5.html Mark Zuckerberg — "Personal Superintelligence" — https://www.meta.com/superintelligence/ Resolution — the theory-plus-empirics alignment org Adam is excited about — https://resolution.org/ Rationality: From AI to Zombies — Eliezer Yudkowsky's Sequences — https://www.readthesequences.com/ Robin Hanson — Futarchy: Vote Values, But Bet Beliefs (prediction markets as decision processes) — https://mason.gmu.edu/~rhanson/futarchy.html The OpenAI–Hugging Face Incident — original Black Hat USA 2026 talk — https://www.youtube.com/watch?v=87DyyMV0kCY OpenAI's Model Just ATTACKED Them — the Hugging Face hack breakdown — https://www.youtube.com/watch?v=RczYubQzXbI Robin Hanson vs. Liron Shapira: Is Near-Term Extinction From AGI Plausible? — https://www.youtube.com/watch?v=dTQb6N3_zu8 Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

  4. Aug 28

    Sam Altman Is Gaslighting About AI Risk After His Own AI Just Went Rogue

    Sam Altman keeps insisting that AI progress is going better than the doomers predicted and that even superintelligence may not change the world as radically as people think. I react to his latest interview and explain why I think that calm, reassuring framing badly downplays the danger we're actually in. I go through the interview line by line: Sam's "frame control", his framing of AI as normal technology, his "pro-human" branding, the liberty-vs-safety pivot, and his victory lap on AI safety — all while his own AI just went rogue. Don't let them pull the Overton window backward. Watch on YouTube: https://www.youtube.com/watch?v=yb9lNGHVycs Timestamps 0:00 Teaser 0:47 Why I’m reacting to Sam’s interview 3:17 Sam Altman’s “frame control” 5:51 Framing AI as normal technology 9:50 Let’s watch Sam do it 10:34 “The world… won’t be that different” with superintelligence 12:25 This is gaslighting 12:30 Sam acknowledges loss of control 15:16 His other big risk: centralized power 19:44 Sam’s “pro-human” framing 25:33 Conflating AI critics with anti-human views 30:13 “Liberty vs. safety” 33:50 Sam vs. the “doomers” 36:53 Is alignment really an “unsolvable problem”? 38:00 Sam says the doomers predicted wrong 46:00 “The crazy bad predictions… have not happened” 47:06 Sam’s lean-startup theory of AI safety 50:26 Taking a victory lap on AI safety 55:43 “Disconnecting yourself from reality” 59:11 What would actually make OpenAI slow down? 1:03:19 Sam compares AI safety to aviation safety 1:07:30 Charging into the fog 1:12:50 The “missing mood” around AI extinction 1:14:51 Richard Ngo on Sam Altman’s “earnestness field” 1:19:01 Don’t let them pull the Overton window backward Links Sam Altman on David Senra — “Sam Altman on Building OpenAI & Betting on the Impossible” (the interview I react to) — https://www.youtube.com/watch?v=kG8AoExkX40 Richard Ngo — What Just Happened? Pragmatism and Pessimization (the “earnestness field” post) — https://www.lesswrong.com/posts/yaz8nx4ogZmiqHzt7/what-just-happened-pragmatism-and-pessimization Richard Ngo — What Just Happened? A Retrospective of AI Alignment (start of the series) — https://www.lesswrong.com/posts/9RL9MuGZjzm4q3gKG/what-just-happened-a-retrospective-of-ai-alignment Eliezer Yudkowsky & Nate Soares — If Anyone Builds It, Everyone Dies — https://www.amazon.com/dp/0316595640 Eliezer Yudkowsky — Coherent Extrapolated Volition — https://www.lesswrong.com/w/coherent-extrapolated-volition Doom Debates: Dario Amodei BUNGLES Another Essay — MIRI’s Harlan Stewart Reacts — https://www.youtube.com/watch?v=aCYVVzza7A0 Doom Debates: OpenAI’s Bombshell Hack — Swarms of Agents — https://www.youtube.com/watch?v=RczYubQzXbI Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

  5. Aug 26

    They’re Making AI Doom Cool! Ft. AELLA, Brangus, Avalon Warren, Avisha NessAiver & Josh Thor of PlzDontKillUs

    Come with me into the world of PlzDontKillUs, a bold new AI x-risk communication accelerator program that just finished its first cohort. PlzDontKillUs took the influencer-house formula and moved it to Berkeley, giving creators personal mentorship from AI safety experts like Eliezer Yudkowsky and Nate Soares. With 57 people making daily videos for a month, the program racked up over 100 million views. Find out how Aella, an independent sex researcher, and Ronny Fernandez, a generalist at Lightcone Infrastructure, teamed up to run the largest ever creator bootcamp focused on AI x-risk reduction. Then meet three of the program’s top creators: Avalon Warren, Avisha NessAiver, and Josh Thor. We do a post-mortem on PDKU and ask: Was it effective? What are creators taking from it, and do they have complaints? And does doomerism really boost view counts? PlzDoEnjoyOur special episode on PlzDontKillUs! Watch on YouTube: https://www.youtube.com/watch?v=aPsjMCG7Ic8 Timestamps 00:00:00 — Cold Open 00:01:19 — Introducing PlzDontKillUs & Aella 00:09:33 — Aella Being Early to AI Doom 00:14:45 — The Origin of PlzDontKillUs 00:19:08 — How It Went 00:25:40 — Complaints with Prize $$ 00:29:33 — Aella, What's Your P(Doom)?™ 00:31:14 — Transhumanism & AI Girlfriends 00:34:38 — Aella Rides the Doom Train™ 00:38:08 — Introducing Ronny Fernandez 00:41:21 — Running Lighthaven 00:47:18 — Life Inside PDKU 00:52:35 — Smashing the Overton Window 00:59:27 — Ronny, What's Your P(Doom)?™ 01:05:41 — Brangus Lore, No Cringe, No Disgust 01:16:57 — Introducing Avisha NessAiver 01:17:44 — How He Makes Viral Hits 01:22:16 — Award-Winning Explainer Vid 01:26:51 — Avisha, What's Your P(Doom)?™ 01:32:11 — Introducing Avalon Warren 01:35:49 — Normalizing AI Doom 01:37:53 — Avalon, What's Your P(Doom)?™ 01:39:27 — Introducing Josh Thor 01:41:36 — Josh, What's Your P(Doom)?™ 01:44:34 — Josh's Hits: Cover Song & Soares Collab 01:47:50 — Josh's Critiques of PDKU 01:52:05 — Cold-Emailing a Politician 01:54:07 — Donation Drive 01:55:01 — Outro: What Are You Fighting For? Links PlzDontKillUs (applications open for round two) — https://pleasedontkillus.com Aella on X — https://x.com/Aella_Girl Aella's Substack (Knowingless) — https://aella.substack.com Ronny Fernandez's Substack (Brangus) — https://ratorthodox.substack.com Ronny Fernandez on X — https://x.com/RatOrthodox Avisha NessAiver — Distilled Science on Instagram — https://www.instagram.com/distilledscience/ Distilled Science (site) — https://distilledscience.xyz Avalon Warren on Instagram — https://www.instagram.com/avalonwarrenn/ Josh Thor on X — https://x.com/joshthor9 Josh Thor — "Last Year Alive" ("Last Friday Night" AI apocalypse parody, ft. Eliezer Yudkowsky and a Nate Soares sax solo) — https://www.youtube.com/watch?v=9fYIm72GqrE Tyler Alterman — "What are you fighting for? AI safety professionals answer" (the closing montage) — https://www.youtube.com/watch?v=9sN1VfVAp6o Nate Soares — "AI Danger But You Don't Understand Full Sentences" (caveman video, script by Josh Thor) — https://www.instagram.com/p/DboGRisxC2E/ Avisha's AI water-use misinformation debunk (8.5M views) — https://www.instagram.com/p/DbXV6j9k8JE/ Avisha's award-winning "clearest AI x-risk explainer" — https://www.instagram.com/p/DbZeRT4gIq0/ Avisha's viral "Tingles and Touch" oxytocin video — https://www.instagram.com/p/DIxMLxkuhCl/ Avalon — "if the AI apocalypse is coming, might as well dress cute" — https://www.instagram.com/p/DbUi6yGCS9M/ Avalon — warrior-princess ballroom dress video (3.6M views) — https://www.tiktok.com/@avalonwarren/video/6892040140248632582 Serialsevens — the anime "If Anyone Builds It" video — https://www.instagram.com/serialsevens/ Kenorlands — PDKU creator clip featured in the episode — https://www.instagram.com/kenorlands/ Evie (@0xgingergirl) — "first month in SF" video — https://x.com/0xgingergirl White House press briefing, March 30, 2023 — Peter Doocy asks about "literally everyone on Earth will die" and Karine Jean-Pierre laughs — https://bidenwhitehouse.archives.gov/briefing-room/press-briefings/2023/03/30/press-briefing-by-press-secretary-karine-jean-pierre-22/ White House press briefing, May 30, 2023 — the AI extinction statement gets a serious answer — https://bidenwhitehouse.archives.gov/briefing-room/press-briefings/2023/05/30/press-briefing-by-press-secretary-karine-jean-pierre-and-office-of-management-and-budget-director-shalanda-young/ Wes & Dylan Join Doom Debates — Violent Robots, Eliezer Yudkowsky, & Who Has the HIGHEST P(Doom)?! — https://www.youtube.com/watch?v=Fy6gz41-6gc Aella — "My attempts to sensemake AI risk" (Aug 2022) — https://aella.substack.com/p/my-attempts-to-sensemake-ai-risk If Anyone Builds It, Everyone Dies — https://ifanyonebuildsit.com --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

  6. Aug 20

    I Warned You AI Would Hack Us — My 2023 Interview Aged Terrifyingly Well

    In April 2023, weeks after GPT-4 launched, when many were saying it was just a "stochastic parrot", I went on AdQuick's Madvertising podcast hosted by Adam Singer and warned that GPT was "about to be the strongest hacker the world has ever known". Watch that conversation, which also covered the Web3 bubble and AI doom more broadly, and judge for yourself how my claims are holding up. Watch on YouTube: https://www.youtube.com/watch?v=M6gNwZWY-T8 Timestamps 00:00:00 — Adam Introduces Liron 00:02:03 — Liron's P(Doom) Poll 00:03:24 — Web3 Postmortem 00:05:05 — Axie Infinity and Hollow Abstractions 00:07:54 — Brand Marketers' Metaverse FOMO 00:13:33 — Near-Term AI: Elevator to Heaven 00:16:46 — AI at Relationship Hero 00:18:25 — From Chatbots to Bricking the Universe 00:20:17 — The Passive Income Doom Scenario 00:22:58 — 50 Orders of Magnitude Smarter 00:26:10 — Accelerationism and Transhumanism 00:28:51 — Why There's No Off Button 00:31:00 — Can't We Hard-Code Safeguards? 00:33:13 — CRISPR, Nukes, and Fusion 00:35:40 — AI Doom vs. Climate Change 00:39:10 — Nanotechnology Endgame 00:41:45 — AI Attackers vs. AI Defenders 00:43:28 — The Ethics of AI Persuasion 00:46:41 — The Blue Sky Scenario 00:49:04 — Will AI Kill Craft? 00:51:13 — Stuck Culture and Hollywood Sequels 00:54:50 — The Pause Letter: Don't Train GPT-5 00:59:00 — Worldcoin and Proof of Humanity 01:00:44 — "You Probably Won't Die of Old Age" 01:01:52 — Wrap-Up Original episode: https://madvertising.buzzsprout.com/2118560/episodes/12603815-liron-shapira-web-3-mania-cybersecurity-how-ai-could-brick-the-universe-madvertising-6 Links ADAM SINGER & MADVERTISING Adam Singer on X: https://x.com/AdamSinger Madvertising by AdQuick (the original show): https://madvertising.buzzsprout.com/ LIRON'S PROJECTS MENTIONED Bloated MVP (Liron's blog on hollow abstractions): https://www.bloatedmvp.com/ REFERENCED IN THE EPISODE "Pause Giant AI Experiments" open letter (FLI, March 2023): https://futureoflife.org/open-letter/pause-giant-ai-experiments/ Eliezer Yudkowsky — "Pausing AI Developments Isn't Enough. We Need to Shut it All Down" (TIME, March 2023): https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ Rob Miles AI Safety on YouTube: https://www.youtube.com/c/robertmilesai Engines of Creation by Eric Drexler: https://www.amazon.com/Engines-Creation-Nanotechnology-Library-Science/dp/0385199732 --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

  7. Aug 18

    Former Singularity Institute President: PauseAI Is Making AI Doom WORSE

    Michael Vassar used to work closely with Eliezer Yudkowsky as President of the Singularity Institute for Artificial Intelligence, the organization that later became the Machine Intelligence Research Institute (MIRI). Even though his P(Doom) is in the tens of percents by 2050, he says the people trying to pause AI are "bad people”. He hopes this episode convinces me to stop being a fearmonger… but I claim the general public needs to hurry up and get more scared. All aboard the Doom Train! 🚂 Watch on YouTube: https://www.youtube.com/watch?v=4GbFng6htaQ Timestamps 00:00:00 — Cold Open 00:01:26 — Introducing Michael Vassar 00:02:39 — Shorting Boeing at the Start of COVID 00:05:41 — What's Your P(Doom)™? 00:08:11 — The Magic Dial Thought Experiment 00:10:25 — "A Treaty Between Demis and Dario" 00:13:33 — AI CEOs 20 Years Ago 00:15:59 — Lessons from Peter Thiel 00:17:47 — Riding the Doom Train™ 00:19:41 — Will the Human Brain Stay Relevant? 00:22:21 — When Does a Singleton Form? 00:28:35 — Hugging Face Hack 00:30:43 — Mistake Theory vs. Conflict Theory 00:33:18 — Debating the Orthogonality Thesis 00:36:38 — Maximally Evil AI Scenario 00:40:28 — Is the AI Development Process Safe? 00:42:05 — "PauseAI People Are Bad People" 00:44:44 — Michael Psychoanalyzes Liron 00:49:39 — How Liron Can Redeem Himself 00:55:10 — Michael Critiques Eliezer Yudkowsky 00:59:11 — Is Doomerism Bad Epistemology? 01:02:03 — Do Serious People Believe in Doom? 01:04:45 — Should Liron Stop Fearmongering? 01:07:16 — Political Violence Won't Help 01:11:04 — The Data Center Airstrike Scenario 01:13:34 — Could Every Nation Sign a Pause Treaty? 01:17:21 — What "Governance" Actually Means 01:26:12 — Michael's History with Rationalists 01:33:43 — Are We in a Computer Simulation? 01:38:43 — Wrap-Up Links Michael Vassar on X — https://x.com/HiFromMichaelV The anti-anti-normativity sequence on Natural Hazard (Michael's recommended reading) — https://naturalhazard.xyz/ben_jess_sarah_starter_pack Michael on DemystifySci #366 — "1000 Year Plan to Crush Thought" — https://www.youtube.com/watch?v=W17Cx21in9I Michael on The Vance Crowe Podcast #303 — Good vs. Evil — https://open.spotify.com/episode/2RzlQDSwxGbjloRKqCh1xg Michael on Clearer Thinking with Spencer Greenberg — Preference Falsification and Postmodernism — https://podcast.clearerthinking.org/episode/028/michael-vassar-preference-falsification-and-postmodernism/ "The Survival of Noise Traders in Financial Markets" — De Long, Shleifer, Summers & Waldmann — https://www.nber.org/papers/w2715 Scott Alexander — Conflict vs. Mistake — https://slatestarcodex.com/2018/01/24/conflict-vs-mistake/ Scott Alexander — Kolmogorov Complicity and the Parable of Lightning — https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/ Scott Alexander — Samsara — https://slatestarcodex.com/2019/11/04/samsara/ Eliezer Yudkowsky — Meta-Honesty: Firming Up Honesty Around Its Edge-Cases — https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases Zvi Mowshowitz — Book Review: Going Infinite — https://thezvi.substack.com/p/book-review-going-infinite Jaan Tallinn — "Why Now? A Quest in Metaphysics" (Singularity Summit 2012) — https://vimeo.com/54718573 Paul Graham — "Five Founders" (the Sam Altman essay) — https://paulgraham.com/5founders.html Truthful AI (Owain Evans' safety org) — https://truthful.ai METR — Measuring AI Ability to Complete Long Tasks (the time-horizon graph) — https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ Statement on AI Risk (Center for AI Safety, 2023) — https://safe.ai/work/statement-on-ai-risk Vitalik Buterin — Will "d/acc" Protect Humanity from Superintelligent AI? — https://lironshapira.substack.com/p/debate-with-vitalik-buterin-will Holly Elmore — These Effective Altruists Betrayed Me — https://www.youtube.com/watch?v=mFFObQ6vnLU Noah Smith — Top Economist Says P(Doom) Is 0.1% (the "AI drugs" claim) — https://www.youtube.com/watch?v=AwmJ-OnK2I4 --- Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

  8. Aug 11

    14-Year-Old Forecaster Challenges My P(Doom) — Eli Goldfine

    Eli Goldfine is an unusually thoughtful 14-year-old podcast host who’s rapidly becoming an authoritative voice in the prediction market space. In this unique episode, Eli brings a critical yet open-minded approach to the topic of AI doom that we older folks could learn from.We cover what it's like to grow up in the AGI era, and Eli's hobbies which include reading rationalist bloggers, betting on prediction markets, vibe coding with Claude, and developing a software startup for astronomers.In our debate, Eli expects superintelligence by 2032, but argues P(Doom) can't be estimated. I claim you can't opt out of Bayesian epistemology. All aboard the Doom Train! 🚂 Watch on YouTube: https://www.youtube.com/watch?v=Bqk5vqpzG_o Timestamps 00:00:00 — Cold Open 00:01:07 — Introducing Eli Goldfine 00:08:31 — Interviewing Robin Hanson at Manifest 00:10:55 — The Supercycle Podcast 00:14:37 — Eli's Betting Market Scandal 00:16:49 — Will Eli Graduate High School? 00:18:27 — Decentralized Telescope Software 00:26:50 — Vibe Coding Without CompSci Basics 00:32:08 — Robin Hanson, Futarchy, MetaDAO 00:40:56 — What's Your P(Doom)?™ 00:46:32 — Liron Defends His 50% P(Doom) 00:51:00 — Should P(Doom) Inform Policy? 00:59:16 — What's Your P(Bloom)? 01:03:04 — Pascal's Wager for Accelerationists 01:04:04 — Donation Drive 01:11:22 — Are E/ACCs Incredibly Dumb? 01:13:42 — How Powerful will ASI Become? 01:16:20 — Predicting ASI by 2032 01:23:56 — Let's Ride the Doom Train™ 01:35:26 — Orthogonality, Culture, and Moral Realism 01:44:39 — Can Safety Keep Pace with Capabilities? 01:49:12 — Debating Instrumental Convergence 01:54:40 — Culture, Laws, and the Singleton Scenario 01:59:53 — Ricardo's Law Won't Save Us 02:03:32 — The Case for Racing China 02:05:40 — Wrap-Up & Ideological Turing Test 02:10:36 — Outro from Conductor Ori Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe

See All (189)

Ratings & Reviews

4.1
out of 5
21 Ratings

About

It's time to talk about the end of the world. With your host, Liron Shapira. lironshapira.substack.com

Information

  • Creator
    Liron Shapira
  • Years Active
    2024 - 2026
  • Episodes
    189
  • Rating
    Explicit
  • Copyright
    © Liron Shapira
  • Show Website
    Doom Debates!

You Might Also Like