LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

  1. -1 h

    “Debate with Self-Play Best-of-N Optimization” by Dewi Gould, Sam Martin, Alejandro Aristizabal, Simon Marshall, Jacob Pfau

    Debate is a proposed protocol for scalable oversight. As tasks outrun direct supervision, labs are increasingly likely to train against protocols like it. Our concern is that, for questions which are hard to verify, models will become more compelling more quickly than they will become more accurate – this could undermine alignment research and safe use. Whilst existing public empirical work mostly focuses on debate as an evaluation protocol (does debate help a judge reach better verdicts?), there is limited work using debate as a reward signal for training. This note is the first in a series aimed at building an open, empirical science of debate training. We show that inference time optimization, via best-of-N (BoN), can be used to iterate on debate protocols – de-risking training runs before committing to RL. By building up a careful, controlled understanding of how optimization pressure interacts with protocols, we lay the groundwork for tackling higher-level questions with confidence. We introduce an inference-time proxy for debate training. Studying debate protocols using BoN allows us to scale optimization on different players independently and identify which parts of the debate game are doing work. We believe that BoN provides sufficient optimization power to [...] The original text contained 9 footnotes which were omitted from this narration. --- First published: July 9th, 2026 Source: https://www.lesswrong.com/posts/hb8pv3zyAHGJpwz9F/debate-with-self-play-best-of-n-optimization --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  2. -1 h

    “How big is the Sun? How could you figure it out?” by Elliott Thornley

    I’m a few chapters into Our Mathematical Universe by Max Tegmark. By this point he's covered the ingenuities of the ancient Greeks, taking my knowledge of physics to within two and a half thousand years of the cutting edge. And what ingenuities they were. A whole series of them, strung together, and culminating in a pretty good estimate of the size of the Sun. I think it's a remarkable feat to even just ask the question ‘How big is the Sun?’ and recognize that it has an answer. The fact that the ancient Greeks actually managed to figure it out (more or less) is astonishing. I don’t know the exact path they took with their reasoning. History is messy. But here's a plausible route they could have taken, basically matching the one you’ll find charted in OMU. Step 1: Discover that the Earth is round. One way to discover that the Earth is round is to wait for a lunar eclipse, where the Earth casts its shadow on the Moon. If you look carefully, you’ll notice that the shadow is curved. Another sign of Earth's roundness is the way that departing ships disappear over the horizon: hull-first and mast-last. [...] --- Outline: (00:57) Step 1: Discover that the Earth is round. (01:37) Step 2: Use the disappearing-ship trick to estimate Earth's size. (02:24) Step 3: Use Eratosthenes' shadow trick to estimate Earth's size again. (03:43) Step 4: Use the similarity of your estimates to surmise that the Sun is really far away. (04:51) Step 5: Use the Sun's distance and apparent size to infer that it's bigger than the Earth. (06:02) Step 6: Use the Earth's shadow on the moon to estimate the Moon's size. (08:15) Step 7: Use the Moon's real size and apparent size to estimate its distance. (08:58) Step 8: Use the Moon's distance to estimate the Sun's distance. (10:24) Step 9: Use the Sun's distance and apparent size to estimate its real size. (10:56) Epilogue The original text contained 3 footnotes which were omitted from this narration. --- First published: July 9th, 2026 Source: https://www.lesswrong.com/posts/Bc4Ch63cx8KHdQABw/how-big-is-the-sun-how-could-you-figure-it-out --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

    12 min

  3. -3 h

    [Linkpost] “AI 2040: Plan A” by Daniel Kokotajlo, elifland, Thomas Larsen, romeo, bhalstead, ryan_greenblatt

    This is a link post. For the past year, we at the AI Futures Project have been sinking most of our time into our next big scenario. Now it's done! It's called AI 2040: Plan A. It's called Plan A because it's a recommendation, not a prediction. It's what we think should happen, not what will happen, though we think it's plausible enough to aim for. It's called AI 2040 because in it, they delay the creation of superintelligence to 2040. It would have happened much sooner (in 2030, to be precise) if not for decisive action on the part of the US and Chinese governments. As with AI 2027, summaries don’t really do it justice, since the whole point was to be detailed and comprehensive and work things out step by step rather than rely on high-level abstractions like doom or utopia. Read the scenario at ai-2040.com. You can listen to it on audio, or view it on mobile, but the experience is significantly better on a normal computer. What's next for us? Well, first we are going to respond to comments and otherwise engage with whatever conversation, responses, critiques, etc. that [...] --- First published: July 9th, 2026 Source: https://www.lesswrong.com/posts/pFzctpJBat95SrCyC/ai-2040-plan-a Linkpost URL:https://www.ai-2040.com/ --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

    2 min

  4. -5 h

    “Announcing our $160M grant from Coefficient Giving” by Geoffrey Irving, Jesse Hoogland, Alex HT, Jacob Pfau, Daniel Murfet, Marco Cozzi, Stan van Wingerden

    We are excited to announce that Resolution (fka Sequent) has a 160 million dollars grant from Coefficient Giving (cG) to put rigorous alignment research on a (closer to) even footing with the frontier labs. We will use it to accelerate progress towards higher-confidence alignment, or to find evidence and obstacles showing why alignment is hard. The grant is structured as a 108 million dollars base plus 52 million dollars conditional on a combination of hiring success and compute needs. The base includes a small regranting budget, which we plan to use both for high-quality non-Resolution alignment research and to give back to shared community infrastructure that we depend on. Coefficient Giving will be our sole funder to start (thank you!); our goal is to raise larger-scale funds from a mixture of sources once we demonstrate success with semiautomated alignment theory and empirics. The time for ambitious alignment funding is now In our launch announcement, we argued that the time to automate alignment is now: frontier systems have finally reached the threshold where they can make nontrivial theoretical progress, and theoretical research has access to more sources of ground truth than empirical metric climbing. There is a second, structural [...] --- Outline: (01:09) The time for ambitious alignment funding is now (02:20) Please join us in speeding up higher-confidence alignment! (03:38) Join us in closing the gap --- First published: July 9th, 2026 Source: https://www.lesswrong.com/posts/HDKQNqiR2gtfMiWsn/announcing-our-usd160m-grant-from-coefficient-giving --- Narrated by TYPE III AUDIO.

    5 min

  5. -6 h

    “Because 8 ≈ e², Anthropic’s researcher uplift is plausibly >2x” by Thomas Kwa

    Note: the modeling assumptions and conclusion are Thomas Kwa's opinion, and others at METR disagree. [1] Also, the math was checked by Claude but not a second human. Introduction Anthropic's RSI blog post reported that in Q2 2026, Anthropic contributors merged 8× as much code per day as in the 2021-2024 period. What does this imply about the factor by which a researcher's total effective output increased — the (serial) researcher uplift [2] ? Of course, 8× more code doesn't mean 8× more research, as coding is only part of the job. However, if we assume each line of code (LoC) has equivalent quality and verbosity to pre-2025 code and make standard economic modeling assumptions, we can conclude a surprising amount: all models predict researcher uplift at Anthropic from coding agents alone is >2×. (Researcher uplift could be even higher, because these numbers assume no uplift on non-code tasks.) Cobb-Douglas predicts that if pre-AI time spent coding is and code output increases by a factor , then researcher uplift is . CES (constant elasticity of substitution) production functions, due to a fun mathematical coincidence, infer a narrow range of about if code is [...] --- Outline: (00:25) Introduction (03:09) Economic models (03:57) Cobb-Douglas model (04:57) CES model (05:26) Setup (06:31) Because 8 ≈ e², the estimate is robust to σ! (09:15) M = 8 also implies a lower bound on σ (10:47) Code heterogeneity model: What if AI speeds up high-stakes code less? (12:22) Caveats: How could Anthropic's uplift be less than 2x? (12:36) Plausible reasons (12:39) Verbosity (14:42) Barely useful code (16:14) Irrational time allocation, intrinsic desire to use AI, etc. (17:17) Unlikely reasons (17:21) Extreme heterogeneity (18:04) Changing denominator (18:34) Discussion (18:37) Prefer code output over code uplift, for estimating overall uplift (19:40) Why is Anthropic's own estimate much lower? (21:55) There are reasons to expect even higher uplift (22:46) Conclusion (24:03) Appendix: Heterogeneous code model The original text contained 8 footnotes which were omitted from this narration. --- First published: July 9th, 2026 Source: https://www.lesswrong.com/posts/ix5qEyW9BjGEb4d8k/because-8-e-anthropic-s-researcher-uplift-is-plausibly --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

    26 min

  6. -6 h

    “Optimiser Choice Can Amplify or Suppress Emergent Misalignment” by Jason R Brown, Patrick Leask, Lev McKinney

    This is a linkpost for https://arxiv.org/abs/2606.31591. Work done with Patrick Leask and Lev McKinney during the Astra Fellowship. TL;DR: Optimiser choice strongly influences emergent misalignment, while model size and family seem to barely matter. Optimisers that concentrate the LoRA update into fewer directions degrade alignment more, but regularising towards a flatter spectrum can mitigate this and improve alignment. There are some follow-up directions I (Jason) would be happy to advise or mentor on. Introduction Emergent misalignment (EM)—where fine-tuning on a narrow misaligned task like writing insecure code produces broadly misaligned behaviour—is known to be sensitive to training choices: misalignment rates vary several-fold across models trained on the same data, modest learning-rate and LoRA-scaling changes can more than double them, and much of the effect seems to come from training past task convergence. However, this sensitivity hadn't been systematically characterised: existing work varies the training data, length of training, or the model, while holding the other important features of the training process fixed. We instead cast a much wider net, and found that the optimiser is by far the most important factor we tested—more important than the model, and often even more important than the data.[1] What we found Model [...] --- Outline: (00:44) Introduction (01:34) What we found (04:41) Future directions (07:39) Acknowledgements The original text contained 4 footnotes which were omitted from this narration. --- First published: July 9th, 2026 Source: https://www.lesswrong.com/posts/Wq6CaAbiixoCEzbat/optimiser-choice-can-amplify-or-suppress-emergent-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

    9 min

  7. -10 h

    “Childhood and Education #20: Phones and Screens” by Zvi

    We have a respite, so I thought I’d tackle various thoughts on children, phones and screens. GPT-5.6-Sol drops tomorrow, and the Fable agents are hard at work. I’ll start with the other screens, then finish with the phones. Table of Contents EdTech. NonEdTech. Do Not Ban Social Media Outright. Some Modern Kids Media Is Pretty Great. Ban Phones In Schools (1). Your Offer Is Acceptable. Ban Phones In Schools (2). Screen Time. Inappropriate Content. EdTech Increasingly, when you pick a school, you are picking EdTech. The school will put your child on a tablet or computer, and expect them to learn that way. In theory, with sufficient assistance and bespoke design and incentive structures, this is The Way. It sure seems way better than ‘sit and listen to a lecture.’ I am especially excited for Alpha School's version of this, with its bespoke designs and high level of both expectations and continuous human support. Alas, most people are getting a much worse version, that is much worse than what you could easily improvise at home. I’m less concerned with ‘EdTech provider [...] --- First published: July 8th, 2026 Source: https://www.lesswrong.com/posts/XA8fMCnwuc45uZYHX/childhood-and-education-20-phones-and-screens --- Narrated by TYPE III AUDIO.

  8. -15 h

    “How slower does takeoff go with 10× less compute?” by bhalstead

    About 6x slower in the median case, with an 80% confidence interval of 3.5x to 8x. Setup Define the "R&D compute" (in, say, H100-equivalents) of an AGI company at a given time to be the total compute in use across the following categories: Compute used to run experiments, which are used to search for software improvements,Compute used to run automated researcher agents,Compute used in final training runs for frontier models. With less R&D compute, the company would need to make cuts across these categories. For now, we'll assume they maintain the same allocation across the three areas, that is, we'll assume they achieve the reduction in the total via a reduction to each category. The capability level of your best AI model is represented by its effective training compute, . is a stock, with units of "2025-FLOP". evolves according to where is training system performance (a flow, units: FLOP/yr) and is software efficiency (units: 2025-FLOP per FLOP). We will call this quantity effective training system performance. Training system performance and software efficiency tend to grow exponentially over time, so the "rate of AI progress" will be operationalized as the relative growth rate of (i.e. the [...] --- Outline: (00:30) Setup (02:57) Finding a mathematical expression for the slowdown factor (06:25) Estimating using the AI Futures Model The original text contained 1 footnote which was omitted from this narration. --- First published: July 8th, 2026 Source: https://www.lesswrong.com/posts/7jcPg79p3kD5ir3CL/how-slower-does-takeoff-go-with-10-less-compute --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

    8 min
Tout afficher (250)

À propos

Audio narrations of LessWrong posts.

Informations

  • Création
    LessWrong
  • Années d’activité
    2023 - 2026
  • Épisodes
    250
  • Classification
    Tous publics
  • Copyright
    © 2026 All rights reserved
  • Site web de l’émission
    LessWrong (30+ Karma)

Vous aimeriez peut‑être aussi