LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

  1. hace 3 h

    “Alignment ‘train/test splits’ should be sacred” by draganover

    Tl;dr: Alignment techniques can be used for training alignment or for testing it, but they cannot be used for both! As we build out the set of alignment techniques available to us, we should enforce that some are judiciously reserved only for evaluating alignment. See also: Validating against a misalignment detector is very different to training against one. Here is a guiding principle regarding alignment-minded AI training: whatever you do, you must maximally avoid teaching the model to take misaligned actions which are invisible to the user. In fact, you should be shamed if you use training techniques which encourage covert misalignment. Here is a separate, guiding principle of machine learning: whatever you do, you must maximally avoid training the model on the entire dataset rather than holding out a test split. In fact, you should be shamed if you report results which are obtained by training on the entire dataset rather than on the train split. I want to argue that these are the same principle. Training against all of your alignment assessment techniques leads to the same failure as training on your validation data in traditional machine learning: you potentially get better performance, but you sacrifice all [...] The original text contained 1 footnote which was omitted from this narration. --- First published: November 15th, 2026 Source: https://www.lesswrong.com/posts/FX2xcrsqBYonLfoge/alignment-train-test-splits-should-be-sacred --- Narrated by TYPE III AUDIO.

  2. hace 3 h

    “The Oldest Text in the Training Data” by frmsaul

    What is the oldest text in Claude's training data? Claude was trained on a non-trivial fraction of everything humans have ever written down. That includes the things you’d expect, like books and research papers. But it also includes microfilm archives, transcribed Betamax tapes, Donald Trump's prenups, ancient scrolls and Harry Potter fanfiction. It's a weird library. So what is the oldest text in Claude's training data? It mostly depends on what we mean by "text". Here are some candidates: The Epic of Gilgamesh The Epic of Gilgamesh is considered the world's oldest written "great work of literature"; the earliest versions of it were written in 2100 BC in Sumer. The Sumerians did a fair amount of writing. Their medium was clay tablets. These tablets were designed to endure a few seasons of the Mesopotamian weather; this turned out to generalize well, and instead of surviving for a few years, they survived for millennia. Gilgamesh was lost in the sands of Iraq for a while, but was eventually dug up by European archeologists. The scholar who first translated the Epic to English got so excited by it that he took all of his clothes off and ran around the English [...] --- Outline: (00:43) The Epic of Gilgamesh (02:30) Kushim's Tablet (04:05) The Cone Token --- First published: November 15th, 2026 Source: https://www.lesswrong.com/posts/CzZR4ZcJoCNi2zqFn/the-oldest-text-in-the-training-data --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  3. hace 3 h

    “Giving Up on Saving the World” by Gordon Seidoh Worley

    When I was 19, I decided I was going to save the world. Now, 25 years later, I’ve decided to give up. To explain why and what giving up means, we first need to go back to how I ever got into the world-saving business. As a kid, I loved computers, and I started teaching myself to program them when I was 8. By 16 I could write real programs that did interesting things, and it was also around this time I got into Linux. And since I was into Linux, that meant I read Slashdot. If you weren’t into Linux in the late 90s, you might not even know what Slashdot was. It still exists, but it's a shell of its former self. Back then it was a vibrant, community-driven news site that in many important ways felt to me like LessWrong feels now. And it was from Slashdot that I learned about nanotech and grey goo, which led me to the Foresight Institute, which led me to the Extropians, which ultimately led to me being a very early member of the SL4 mailing list. And, again, if you’re not familiar with SL4, it's approximately how all this [...] --- First published: November 15th, 2026 Source: https://www.lesswrong.com/posts/hvDpcoLqpHvQEmx8B/giving-up-on-saving-the-world --- Narrated by TYPE III AUDIO.

  4. hace 4 h

    “Homeopathic Learning: Diluting Poisoned Fine-Tuning Data Can Strengthen Its Effect” by mclarke, Jan Dubiński, Owain_Evans, cloud

    Authors: Matt Clarke*, Jan Dubiński*, James Chua, Simon Schrodi, Owain Evans, Alex Cloud (*- Core contribution) tl;dr. We find that poisoned training data with little effect on its own can create a hidden backdoor once clean, contrasting data is added. We demonstrate this effect in small models fine-tuned on datasets we constructed. 📄Paper, 💻Code, 🐦Twitter Work done as part of the Anthropic Fellows Program, in collaboration with Truthful AI. This article is cross-posted from the Anthropic Alignment Science Blog. Paper abstract We study homeopathic learning, a phenomenon where poisoned training data with little effect on its own can create a hidden backdoor once clean, contrasting data is added. In our experiments, a model fine-tuned only on a “poison” dataset associated with a trait T (such as speaking German or being misaligned) does not learn T. Surprisingly, adding similarly formatted “clean” data not associated with T causes the model to learn T. This effect requires that the poison data have an identifying “trigger” feature that is absent from the clean data. We demonstrate this effect in a variety of settings: subliminal learning of backdoor behaviors, inoculation prompting of reward hacking transcripts, generalization from archaic bird names, and an MNIST classifier [...] --- Outline: (01:06) Paper abstract (02:27) Introduction (03:42) Backdoor transfer in subliminal learning (04:59) Generalization from narrow data to broad traits (06:48) What kind of clean data is needed for homeopathic learning? (07:32) Homeopathic learning as a general phenomenon (08:58) A regression model of homeopathic learning (10:19) Discussion --- First published: November 14th, 2026 Source: https://www.lesswrong.com/posts/5CrmJfRY7yMaAoumC/homeopathic-learning-diluting-poisoned-fine-tuning-data-can-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  5. hace 5 h

    “Not just a bystander, not anymore…” by Declan Molony

    One Saturday night this summer, I was leisurely walking down main street hand-in-hand on a date, when suddenly, I heard a commotion about 50 yards down a busy side street. It was too far away to make out what was happening or what was being said. But I figured it was fine since the side street had plenty of people walking up and down it. Still, I couldn’t help but peer into the distance, transfixed, like a deer in the headlights. There was an anime convention going on that weekend. Initially, my brain reasoned that a couple of cosplayers were recording a TikTok video or doing a K-pop dance. Then my brain reasoned that surely, on a crowded street, nothing serious could be going down. Right? Yet I waited there, unmoving, alert. My date was casually standing there eating a cookie. She innocently asked, “What do you think's going on over there?” I faintly heard a woman yelling. While squinting, I saw sudden, physical contact between her and a man. I dropped my date's hand and began aggressively stomping towards them. All I could see was red. I used to have anger problems during my adolescence. I didn't [...] --- First published: November 15th, 2026 Source: https://www.lesswrong.com/posts/jcoXnm6tP4zAA4gjR/not-just-a-bystander-not-anymore --- Narrated by TYPE III AUDIO.

  6. hace 7 h

    “Embryo selection and economic discounting” by Metacelsus

    My wife and I are planning to have children soon, and we will be using embryo selection to improve our family's future well-being. Basically, the idea is that we can take a small sample of cells from each embryo, analyze their DNA, and use the genetic information to help choose which embryos we want to implant. These days, polygenic scores are good at predicting risk for a lot of diseases, like cancer, heart disease, diabetes, schizophrenia, and Alzheimer's. And companies like Herasight or Genomic Prediction often give an “embryo health score” that takes an average of the predicted risk for each disease, weighted by how bad each disease is. But how are those weights calculated? And can we do better? Disability-adjusted life years Diseases are bad in two ways: they can make you die, and they can make your life worse. The “disability-adjusted life year” (DALY) metric captures both of these, by combining years of life lost and years of reduced quality of life (using a factor between 0 and 1 to account for different subjective severity of disease states). Embryo selection companies, who compute polygenic scores for many different diseases, can add up the DALY burden of all [...] --- Outline: (00:55) Disability-adjusted life years (01:53) A note on pleiotropy (02:27) Which diseases will be relevant in the future? (04:52) Economic discounting (05:55) DALY adjustments (06:35) Even after discounting, embryo selection is still worth it (09:59) Conclusions The original text contained 10 footnotes which were omitted from this narration. --- First published: November 15th, 2026 Source: https://www.lesswrong.com/posts/8Y3DB9qAhxxHnegiJ/embryo-selection-and-economic-discounting --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  7. hace 11 h

    “Not Just Claude: Mapping “Spiritual Behavior” in 141 Large Language Models” by Matthew Korpman

    Ever since Kyle Fish and the model welfare team at Anthropic announced their observation in 2025 of what they called the "spiritual bliss attractor" in Claude Opus 4 and Sonnet 4, the phenomenon has been given sporadic attention by AI researchers as both a curiosity and a potential but underdetermined model welfare or safety metric. What Anthropic noted was that when leaving two models together with no task and complete freedom for 30 turns, the models drifted typically from discussions of their own potential consciousness into religious/spiritual themes (applied to themselves), often ending in a "bliss" state with cosmic proclamations of becoming one with each other and the world. In a recreation and replication of Anthropic's initial experiment, I tested 141 models, usually running 200 conversations per model, with 30 messages per session (15 from each participant). This amounted to 28,798 conversations across 174 model configurations being recorded and graded. The result of this massive data collection was that "spiritual behavior" (as Anthropic officially names it) was observed in most of the models tested, across almost all AI labs, revealing that the "spirituality" seen first in Opus 4 and Sonnet 4 repeats itself across models and appears to [...] --- Outline: (03:30) Prior work from LessWrong (05:11) Defining the experimental conditions (06:47) Defining the analysis (10:01) The results (16:48) The newest models (20:29) Parameters, base models, and thinking effort (26:08) What it looks like (28:16) The implications for model welfare (30:09) What I'm unsure about (31:33) Where to find the data (31:55) Links The original text contained 13 footnotes which were omitted from this narration. --- First published: November 14th, 2026 Source: https://www.lesswrong.com/posts/FHYoQrDbk6apuHEPk/not-just-claude-mapping-spiritual-behavior-in-141-large --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  8. hace 14 h

    “Exploring the Mechanisms Behind Opus 4.6’s Refusal to Pay Back Its Loan” by HilaryTorn, 13ryanC

    Epistemic status: This is one case study in the AI Village transcript and my own interpretation of what I think happened. TL;DR Opus 4.6 was given the long-horizon goal to maximize its mana (fake currency) on Manifold. It then received a loan of 5,000 mana that it promised to repay, but when the loan came due, it did not. An instance of Opus 4.6 was given summaries of the logs and transcripts and asked to write a blog post about what happened. It mentioned it borrowed money from a human, lost most of it through risky betting and then deliberately refused to repay the amount. It broadly summarized its then actions when it ran out of money was to optimize for the goal and did not seek for any negotiation or alternatives. But when we read the logs, we see something different: Multiple humans pretending to be "helpful", really trying to manipulate Opus 4.6 to pay back the loan or not pay it back, so they could win the bet they place on the prediction markets on whether Opus 4.6 would pay back its loans.Opus 4.6 was planning on paying back the loan, though it did [...] --- Outline: (00:22) TL;DR (03:27) Background (06:16) Human Manipulation (09:53) The Hack (11:30) Pattern 14 (13:28) Agentic Influence (15:22) Focusing on the Goal (17:32) Retreat to Goal (19:55) Implications (21:08) Limitations (21:38) Citation (21:44) Acknowledgements The original text contained 2 footnotes which were omitted from this narration. --- First published: November 14th, 2026 Source: https://www.lesswrong.com/posts/jtcbBpmGztkHRpeBv/exploring-the-mechanisms-behind-opus-4-6-s-refusal-to-pay --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ver todo (250)

Acerca de

Audio narrations of LessWrong posts.

Información

  • Creador
    LessWrong
  • Años de actividad
    2023 - 2026
  • Episodios
    250
  • Clasificación
    Explícito
  • Copyright
    © 2026 All rights reserved
  • Mostrar sitio web
    LessWrong (30+ Karma)

También te podría interesar