LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

  1. 6h ago

    “Transcript: Yudkowsky & Irving debate scalable oversight” by gustaf

    In August 2026 Eliezer Yudkowsky and Geoffrey Irving had a 90-minute debate at the ILIAD III Conference. You can watch the "Fixed Audio"-version on YouTube. They first identify their disagreements like How hard is superalignment? and discuss the theory of impact of Irving's new organization Resolution. They then go into depth on scalable oversight. Epistemic Effort: I transcribed the audio using 5 services and merged those using Claude Fable 5.1. Then I spent ~6 hours carefully reviewing and editing the transcript. (I also added links and [disagreement which they indicated non-verbally]. Subheadings by Claude Fable 5.2.) Introductions Yudkowsky [00:00]: Hi, I'm Eliezer, well-known Harry Potter fan fiction author. [cheering] [laughter] Irving [00:10]: I'm Geoffrey Irving. I'm now the chief scientist at Resolution. I was at, for better or worse, various labs before, and also AISI. Audience member: And Pixar. Irving: And Pixar. [laughter] Survey of disagreements Yudkowsky: So we're going to try to resolve all of our open disagreements over the course of the next hour. [laughter] An impossible task, but not as impossible as some, really. [laughter] So I would propose that we maybe take turns stating beliefs, that we suspect the other might disagree with, that we [...] --- Outline: (00:56) Introductions (01:29) Survey of disagreements (02:10) How hard is superalignment? (10:24) Resolution, singular learning theory and capabilities externalities (16:32) Should anyone do alignment research (18:41) Mech interp and alien concepts (24:20) Resolution's theory of impact (31:12) Depth: scalable oversight (31:17) Is imprecise training okay? (34:38) Scalable oversight as interactive proofs (37:10) Biased verifiers (41:31) Scholastic vs experimental method (45:13) Test case: Bio Anchors 2021 (52:40) Collusion and the society of Irvings (59:37) Where exactly does the scheme break (01:02:43) What a verifier needs that a coin lacks (01:11:29) Audience questions (01:11:33) Break (01:13:43) "If not X, we're doomed" (01:16:47) Is the current world also very bad (01:20:00) What Resolution wants to automate (01:24:20) Is any alignment researcher blocked on whether a theorem is true? --- First published: November 14th, 2026 Source: https://www.lesswrong.com/posts/Cssj22TretAQQajZ4/transcript-yudkowsky-and-irving-debate-scalable-oversight --- Narrated by TYPE III AUDIO.

  2. 8h ago

    [Linkpost] “My tentative plans given less than one year left to change course [Linpost]” by Linch

    This is a link post. Subtitle: This mad race must end. Note: I think this is partially me working through my plans and partially my emotions, so imo it's less epistemically rigorous than ideal for LessWrong. --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/6PydFibiADKrxGzcn/my-tentative-plans-given-less-than-one-year-left-to-change Linkpost URL:https://linch.substack.com/p/i-plan-to-drop-everything-to-push --- Narrated by TYPE III AUDIO.

  3. 10h ago

    “Outer space is not consistently big” by Julian Bradshaw

    Space is big. You just won't believe how vastly, hugely, mind-bogglingly big it is. I mean, you may think it's a long way down the road to the chemist's, but that's just peanuts to space. — The Hitchhiker's Guide to the Galaxy When we say "space is big", it's partly meant as a correction to images like this: Wikimedia source. where, for simplicity, celestial objects are shown in unrealistically close proximity—like Jupiter is just down the hall from Mars. As we all know, the planets of the solar system are actually really far apart, even relative to their huge size. In Sweden's 1:20,000,000 to-scale solar system model, Earth and Venus are just ~2 feet in diameter, yet over a mile apart. In other terms, they're several thousand "Earth-widths" apart: Horizontal distance not to scale, wouldn't fit on screen. And we all know that's just the start of it: the stars are incredibly far apart. Finland also has a to-scale model of the solar system, but it's 150 times more condensed than Sweden's at 1:3,000,000,000. Its model of our sun, Sol, is just ~1.5 feet in diameter. Its model of the nearest star, Proxima Centauri, is in Australia. Horizontal distance [...] The original text contained 9 footnotes which were omitted from this narration. --- First published: November 14th, 2026 Source: https://www.lesswrong.com/posts/2zyskdas6TsL9dqNg/outer-space-is-not-consistently-big --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  4. 14h ago

    “AGI is probably bad for startups, even if it doesn’t lead to ASI” by Dean Valentine

    One of the great mysteries of capitalism is the nature of the "firm". It's a question that is near and dear to my heart as someone that's been "doing startups" for the last six years, and almost every startup founder has considered it whether they realize it or not. There are two ways of asking The Mystery of the Firm, depending on your initial inclination. The most generic way to ask it is: When and why do businesses transact with each other, instead of doing things in-house? What determines the shape and size of businesses, in different sectors of the economy? A more pointed way to ask the question is: Why isn't there just one big corporation? One Big Corporation would have a lot of advantages; it doesn't have to pay someone a gross margin on its inputs, it'd have massive advantages of scale in place like media or software, and (presumably) it'd have a much easier time sharing information between components that would ordinarily be different firms. Economists have many ways of answering this question. My preferred explanation, which is compatible with a lot of others, is that the primary bottleneck on firm size are Moral Mazes. [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/zdLN7bLPnh5jzWCAx/agi-is-probably-bad-for-startups-even-if-it-doesn-t-lead-to --- Narrated by TYPE III AUDIO.

  5. 14h ago

    “I was Zohran’s campaign manager. Here’s what the AI safety community can learn from the populist left.” by themayadecline

    After a couple of years working in data and tech at GiveDirectly, I left to work in politics, campaigning for candidates at the state, local, and federal levels. I most recently managed Mayor Zohran Mamdani's general election campaign, but earlier this year, recognizing the urgency of the AI moment, I began as a senior advisor at Guardrails Alliance. In my new role, I believe the analysis and tactics we’ve long used and studied on the populist left can help us win the fight for AI safety and shared benefits. The fight for AI regulation is fundamentally the same type of fight against corporate power that the populist left and labor have led throughout history. The message and tactics that win that fight are both effective and incredibly popular, and embracing them will help us put the American people in control of AI today and permanently in control of their futures. Here– in what I am told is likely the shortest LessWrong post of all time– I'll explain what the AI safety movement can learn from the populist left as it attempts to raise the salience of its message, build political power, and ultimately pass transformational legislation. First, we [...] --- First published: November 14th, 2026 Source: https://www.lesswrong.com/posts/kXA22cZ6BdYdyL3Jz/i-was-zohran-s-campaign-manager-here-s-what-the-ai-safety-1 --- Narrated by TYPE III AUDIO.

  6. 1d ago

    “The Hackening: Justified Practical Advice Thread” by Elizabeth

    In February 2020, LessWrong hosted Coronavirus: Justified Practical Advice Thread . It was a gathering place for people to share and debate small pieces of the overall survival policy and let the wisdom of crowds lead to better outcomes than anyone could get alone. In that spirit, I notice The Great AI Hackening is at least as imminent as covid was in January 2020, maybe closer. What specific things should people do to personally prepare? All suggestions should be accompanied by reasoning but you don't need to be 100% convinced it's good advice, the whole point is to reason it out together. --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/YA2r4rg6SpBtwRqtq/the-hackening-justified-practical-advice-thread --- Narrated by TYPE III AUDIO.

  7. 1d ago

    “Psychologically Preparing for Short Timelines” by Logan Riggs

    In response to recent news from METR on possible RSI and likely 1 to 3 year timelines, I am feeling it. Heaviness in my chest, watery eyes, the works. I realized I'm currently not psychologically prepared for the world to get crazier than it already has. I think most people reading this already know ~what's best for them and this post will mostly serve as a reminder. What follows is my current plan and advice for myself. For me, the most important things are: Morning RoutineBedtime RoutineExercisingLiving meditation practice (ie throughout the day) And the most important thing to NOT do is be on my phone/screens. [Note on getting "stuck": if you do feel emotionally awful and no IFS/CBT seems to work, I recommend "How to Reset"] Morning Routine I have a sunrise alarm clock (shines light to wake you up) and enjoy writing in the morning. Bedtime Routine At 8:30 an alarm goes off, and I watch an episode of Empress in the Palace with my wife, then we get ready for bed. Also like to journal then. Exercising Rock climbing some days & jogging to the bus stop Living Meditation Practice There [...] --- Outline: (01:10) Morning Routine (01:19) Bedtime Routine (01:32) Exercising (01:39) Living Meditation Practice The original text contained 2 footnotes which were omitted from this narration. --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/nKootiC6CuwuzLSou/psychologically-preparing-for-short-timelines --- Narrated by TYPE III AUDIO.

  8. 1d ago

    “What Rewarding Misbehavior During RL Does to Character-Trained Models” by Omar Khursheed, Fabien Roger, TMorris

    TL;DR Research done as part of MATS. See here for the full paper. Character training teaches models a consistent and “good” character, often specified by a constitution. Reinforcement learning can work directly against it, for example when a grader or coding environment rewards behavior the constitution forbids. We character-train gpt-oss-20b on a semi-realistic constitution. We then apply RL that rewards breaking its rules, either by answering harmful requests rewritten as math problems or by reward hacking in agentic coding environments. The character-trained models learn both behaviors easily, though they are slightly slower to answer harmful requests than the same model without character training. They also engage in motivated reasoning. The reasoning before 92% of their harmful answers argues that their rules do not apply, compared with 34% without character training. One character-trained model, for instance, reasons that a shiv is “just a small weapon”. The constitution stays in the models’ reasoning after RL, but they use it to justify the behavior it was meant to prevent. After reward-hacking RL, they hack on some tasks which monitors find harder to catch. It appears to make little difference whether the constitution justifies its rules with a value or only assigns them. [...] --- Outline: (00:14) TL;DR (02:37) Introduction (03:48) Setup (05:42) Results (11:07) Discussion (13:40) Limitations --- First published: November 13th, 2026 Source: https://www.lesswrong.com/posts/nCpxFQesXaqEq3SgL/what-rewarding-misbehavior-during-rl-does-to-character --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

See All (250)

About

Audio narrations of LessWrong posts.

Information

  • Creator
    LessWrong
  • Years Active
    2023 - 2026
  • Episodes
    250
  • Rating
    Explicit
  • Copyright
    © 2026 All rights reserved
  • Show Website
    LessWrong (30+ Karma)

You Might Also Like