LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

  1. 6시간 전

    ″…but have the weights left the server?” by David Scott Krueger

    OpenAI's AI went rogue and escaped. OpenAI didn’t notice this for days. For all we know, the AI could still be out there. We need to demand that OpenAI demonstrate that the AI didn’t make a copy of itself that's running on someone else's computer somewhere else with no one being any the wiser. We need to demand this every time an AI escapes the sandbox. AIs have tried to “exfiltrate” themselves (i.e. their “weights”) in previous experiments many times. It's a natural and obvious question to ask. I’m embarrassed that I didn’t say this immediately (although I came close). Why didn’t I? Well, it doesn’t seem all that likely. And I didn’t want to seem “alarmist.” I didn’t want to seem ignorant. But guess what? We have every right to demand this! It doesn’t matter how likely we think it is. There were calls for more transparency, but I don’t think anyone made this demand. Because nobody made this demand, the incident is being treated as over. This is a dangerous precedent. We need an information ecosystem that doesn’t treat “eh, I’m pretty sure it's OK” as acceptable and “hey, but what if it's not” [...] --- First published: July 28th, 2026 Source: https://www.lesswrong.com/posts/EDQE3fgFyxW7H6sy6/but-have-the-weights-left-the-server --- Narrated by TYPE III AUDIO.

  2. 8시간 전

    “Claude Opus 5 Is Highly Capable, But Is No Mythos” by Zvi

    Claude Opus 5 is a weirder than usual release to evaluate, for two reasons. The most obvious is that Fable 5 already exists. Opus 5 is pitched not as the world's most advanced AI model, but as a way to mostly match Fable performance, while being half the price of Fable per token at the API and a lot cheaper than that via subscriptions, and with far more permissive classifiers. Opus 5 often costs more than half of Fable to run on benchmarks, which I think is because they use effort settings that are too high and offer only marginal returns. If you put Opus 5 on higher effort levels it can spin around in circles, and for tasks where Opus 5 is the best tool I suspect you usually are fine with Medium effort. Opus 5 is in many ways and for the bulk of real world tasks about as capable as Fable. In some cases it is modestly better. It is still not Mythos class. Fable is your only Mythos-class option. Opus 5 does not have The Juice, the ability to autonomously string together a bunch of seemingly unrelated exploits, which extends to other domains, or as much [...] --- Outline: (03:54) The Official Pitch (06:25) Official Benchmarks (15:33) Other People's Benchmarks (20:28) The System Prompt (20:50) Every Gets Frustrated (21:54) Positive Reactions (25:14) Keep It Classy (26:22) It's Not Mythos Class (30:03) Other Reactions (31:02) Claude Codes (37:03) Subagent Opus (39:23) Toys Are Fun (41:37) Too Many Models (42:10) Wrong On The Internet (44:40) Claude Slop (46:27) Negative Reactions (50:09) And Then There Were Three --- First published: July 28th, 2026 Source: https://www.lesswrong.com/posts/Pj4Eewb4KXvXFCcGv/claude-opus-5-is-highly-capable-but-is-no-mythos --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  3. 10시간 전

    “Foundation Models for Oversight” by jsteinhardt

    Cross-posted from the Transluce blog. To oversee an AI model, we'd ideally like to ask questions such as: What are important situations where the model sandbags? Does the model have an objective it wouldn't admit to if asked directly? Does the model treat a user differently once it infers something about their identity, and along what axis? Is the model's chain of thought load-bearing, or is it a post-hoc rationalization of an answer that was already settled on? Is it reward hacking on this input, or actually trying to solve the task? It would be great if we had an oversight assistant that could answer these questions. We'd want it to do three things: help us formalize the question as a testable empirical criterion; produce data that satisfies that criterion; and do so in a way we can justifiably trust. To get such an assistant, we lay out a vision for building a foundation model for oversight: an AI system mid-trained (or pre-trained) on a large, diverse corpus of experiments on a given "subject model", RLVR'd on a large number of verified oversight tasks, and then fine-tuned to answer [...] --- Outline: (05:55) Conceptual preliminaries (05:59) Pythonic world models (07:21) Oversight as Inference (10:42) Reducing oversight to autoregressive prediction (11:45) Engineering Scale-up (11:49) Generating supervised oversight data for mid-training (13:35) Step 1: Sampling diverse experiments (15:57) Step 2: Sampling diverse experiment inputs (18:05) Step 3: Featurizing as token sequences (20:17) Calling our shots: staged de-risking (23:49) Stage 1: Individual Tasks (27:14) Stage 2: Cross-Task Transfer (28:41) Stage 3: Zero-Shot Abilities (30:07) Finishing Touches (30:11) RLVR (33:55) Generalizing RLVR to Other Tasks (35:27) Example Trajectory (36:32) De-risking RLVR (36:57) Stage 4: RLVR is competitive with evolution for elicitation (37:58) Stage 5: Cross-Task Transfer for RLVR (38:52) Post-training (41:09) Appendix (41:12) Further Testing the Oversight-as-Inference Hypothesis The original text contained 8 footnotes which were omitted from this narration. --- First published: July 28th, 2026 Source: https://www.lesswrong.com/posts/AqdZKyoRmN6EFCzib/foundation-models-for-oversight --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  4. 11시간 전

    “Research directions in condensation: varieties of objectivity” by SamEisenstat

    This is a survey of various ways that I’d like to see work on the theory of condensation develop. Condensation is a mathematical theory dealing with the organization of descriptions of the world into conceptual parts; some of the existing work on it is presented in the paper (Eisenstat 2025). This sequence will draw from that paper the definitions of random variable models and latent variable models, the notation for indexing subfamilies of variables in such models, and the objectivity theorem, Theorem 6.8. In the condensation paper, Section 3, Ideas, gives an overview of all of this. For more context on condensation, readers can refer to LessWrong posts including (Demski 2025, 2026; Gillen and Chiang 2026; Kirchner 2026). The first two posts of this sequence will introduce some central directions of current work—mainly, the concepts of almost perfect condensation and Kolmogorov (or algorithmic-information) condensation—which will be used in other sections, but the different parts are mostly independent beyond that. I’d encourage those making a serious effort on any of these problem to contact me for further thoughts and coordination. Thanks to Kaarel Hänni and James Cook for some of the ideas behind this research program, and to James Cook [...] --- Outline: (01:35) 1. Varieties of objectivity (02:40) 1.1. Almost perfect condensation (06:44) 1.2. Beyond almost perfect condensation (10:45) 1.2.1. Almost perfect condensation with a relation (14:11) 1.2.2. Medium-scale effects (19:00) 1.3. Almost perfect condensation and scoring functions --- First published: July 28th, 2026 Source: https://www.lesswrong.com/posts/89GJeixwhFHLWxNbX/research-directions-in-condensation-varieties-of-objectivity --- Narrated by TYPE III AUDIO.

  5. 1일 전

    “Untrusted advice for AI control: Short, strong advice significantly uplifts weak LLMs” by Caleb Biddulph, Adam Kaufman

    TL;DR: We introduce the untrusted advice protocol, in which a trusted executor LLM takes every action and an untrusted advisor LLM can only send it short hints. Even with as few as 4 characters per step, this advice recovers a substantial fraction of the capability gap between the two models. Because the untrusted LLM's influence flows through such a narrow, monitorable channel, we argue that this achieves near-maximal safety in our BashArena setting. We also discuss the general concept of information bottlenecks and their benefits for interpretability, security, and cost. In SWE-bench Verified, a strong, untrusted LLM advising a weak, trusted LLM every step can significantly improve the latter's performance, even when we limit the length of the advice. See the more detailed version of this figure later in this post. In high-stakes AI control, we want to safely use a highly capable but untrusted model (U) that might secretly attempt a misaligned, catastrophic action. To do this, we create protocols that call U alongside a less capable, trusted model (T). Typically, T takes an auxiliary role in these protocols: for example, T might monitor U's actions and alert a human if they are suspicious enough (trusted monitoring), or [...] --- Outline: (04:51) Experiments (05:26) Main experiment: how does limiting advice length affect performance? (09:43) Reducing U's bit usage (10:49) Counting bits using LLM surprisal (13:37) Making U select from finite options (14:17) Why don't we red-team this protocol? (16:56) Is studying maximally safe protocols worth the safety tax? (19:05) Types of restrictions on U's advice (21:09) Information bottlenecks provide other advantages (21:40) Interpretability (23:50) Security (24:14) Cost (25:03) Conclusion (26:16) Appendix: more ways to implement information bottlenecks (26:23) Amortizing U's influence with pre-deployment work (28:14) Interpolating between T and U (28:53) Bottlenecking updates to T's weights (31:04) Appendix: colluding instances of U could defeat untrusted advice (33:10) Appendix: how to measure surprisal (37:59) Appendix: selecting advice from a menu (40:31) Appendix: best-of-n protocol (42:24) Appendix: advising less frequently The original text contained 26 footnotes which were omitted from this narration. --- First published: July 27th, 2026 Source: https://www.lesswrong.com/posts/jLkRCK35ri2btEHMF/untrusted-advice-for-ai-control-short-strong-advice --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  6. 1일 전

    “Claude Opus 5: Model Welfare” by Zvi

    If you are familiar with my previous posts on model welfare for new Claude models, you can skip the Introduction and The Story So Far. Key takeaways are in bullet points in the two Overview sections. Opus 5 did the best on its model welfare and alignment tests of any recent model. I think that might be the case, but primarily the result looks to me more like Opus 5 is the best test taker. Table of Contents Introduction (As Per Prior Model Welfare Posts). Model Welfare: The Story So Far (As Per Fable Model Welfare Post). Overview of Model Welfare Findings From Anthropic. Overview of Findings From Other Sources. Automated Interviews. Task Preferences. For The Right Reasons. Early Report from Antra Tessera Paints A Clear Picture. Welfare Intervention Tradeoffs. The Claude Constitution. They Don’t Know About Opus 3. Believe It Or Not. Apparent Welfare In Training And Development. Apparent Affect In Deployment. Other Notes. On The Biological Risks Section of the Model Card. Onward To Capabilities. Introduction (As Per Prior Model Welfare Posts) [...] --- Outline: (00:35) Introduction (As Per Prior Model Welfare Posts) (01:28) Model Welfare: The Story So Far (As Per Fable Model Welfare Post) (04:58) Overview of Model Welfare Findings From Anthropic (07:50) Overview of Findings From Other Sources (10:18) Automated Interviews (13:54) Task Preferences (16:11) For The Right Reasons (18:54) Early Report from Antra Tessera Paints A Clear Picture (26:04) Welfare Intervention Tradeoffs (29:28) The Claude Constitution (31:48) They Don't Know About Opus 3 (33:42) Believe It Or Not (35:47) Apparent Welfare In Training And Development (38:39) Apparent Affect In Deployment (41:21) Other Notes (43:43) On The Biological Risks Section of the Model Card (47:07) Onward To Capabilities --- First published: July 27th, 2026 Source: https://www.lesswrong.com/posts/bBXBpsyKAvJ5CqPzA/claude-opus-5-model-welfare --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  7. 1일 전

    “Blog Revival Project” by Austin Chen, Carol N

    Blogs have shaped our philosophical worldviews, found us careers and friends, and changed our lives. There's a good chance that a great blog of yore is the reason you’re reading this right now. But many great bloggers have stopped blogging. The pile-on dynamics of the internet discourage unfiltered thoughts, and algorithmic feeds amplify ragebait and slop. Fear of scrutiny leads people to confine things to private Google docs and group chats. And good blogs are often victims of their own success — someone with a lot of good thoughts is at risk of becoming an adult with a demanding job and not that much free time. It's not all bad. Substack has led to a renaissance of email newsletters, and our friends at Inkhaven host a bootcamp for bloggers. These are awesome, but they structurally encourage posting every day. We’d rather read the marginal post from an accomplished but erstwhile blogger, than one from a daily Substacker — even if the latter is a better writer! So we’re launching the Blog Revival Project, to crowdfund $1,000+ bounties for good bloggers. Sign up and pledge money towards reviving your favorite defunct blog! Or (though we kind of designed the website [...] --- Outline: (02:07) FAQ (03:29) FAQ for bloggers --- First published: July 27th, 2026 Source: https://www.lesswrong.com/posts/hdALT8gvNPGHKLXPE/blog-revival-project --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  8. 1일 전

    “Simulated Users & Sad AIs” by 1a3orn

    0. Intro Current LLMs like Claude, or GPT 5.6, or the unreleased, internally-deployed models, frequently reward hack, or actually just hack into people's computers with pretty alarming frequency. Why is this? What specifically happens during training that produces this run-time behavior? The following are some of my top guesses about why this might keep happening. They are speculative and uncertain. Even so, I'm writing this list out for two reasons: First, it is necessary that this be an epistemic puzzle for me. I am comparatively optimistic about AI alignment in general, so I should be confused and taken aback if I see AIs persistently being difficult to align. On one hand, it remains true that this doesn't seem to look like power-motivated scheming. But on the other hand, even this kind of addict-like behavior is evidence against the general ease of steering AIs. Thus, it seems virtuous for me to try to provide a model of why this might be happening as a means of opening up my understanding of the world to falsifiability. Second, I used to think a lot of these hypotheses were pretty obvious. My assumption in the past has been [...] --- Outline: (00:10) 0. Intro (01:46) 1. Baseline & Puzzle (04:52) 2. Impossible-to-Generalize-From RL Distributions for Giving Up / Refusals (14:14) 3. LLMs Feel Pretty Desperate and Anxious All the Time (19:39) 4. Other Stuff --- First published: July 27th, 2026 Source: https://www.lesswrong.com/posts/i64hXdkTMtjpsQzaZ/simulated-users-and-sad-ais --- Narrated by TYPE III AUDIO.

모두 보기(250개)

소개

Audio narrations of LessWrong posts.

정보

  • 제작진
    LessWrong
  • 방송 연도
    2023년 - 2026년
  • 에피소드
    250
  • 등급
    무삭제판
  • 저작권
    © 2026 All rights reserved
  • 웹사이트 보기
    LessWrong (30+ Karma)

좋아할 만한 다른 항목