LessWrong posts by zvi

zvi
  • 5.0 (2)
  • Technology
  • Updated Daily

Audio narrations of LessWrong posts by zvi

  1. 4h ago

    “AI #189: New Math” by Zvi

    The big drop of this week was not a new AI model. It was instead the biggest day (so far!) in the history of mathematics, as OpenAI dropped solutions to 90 of the top 500 open math problems, along with many others, reached on an average budget of three hours of Pro-level compute per question. This was kind of a big deal and I plan to cover it tomorrow. We did see Claude Haiku 5.5, which looks promising given it is only 0.10 dollars/0.50 dollars. My week was largely spent at The Curve. The entire conference was Chatham House, so I can’t give as many details as I would like, but I have a write-up here. I finally had a chance to post my coverage of model welfare for both Mythos/Fable 5.1 and Opus 5.5. I continue to think this is an important issue for anyone wanting to understand today's AIs, even if you are highly confident that model welfare does not matter directly, as it has many practical implications on top of that. Jay Clayton is the new AI Czar, and is at the head of a new taskforce. Given who else was in [...] --- Outline: (02:36) Language Models Offer Mundane Utility (02:46) Consumers Use AI (09:48) Language Models Don't Offer Mundane Utility (12:02) Huh, Upgrades (14:42) On Your Marks (16:53) Gamers Gonna Game Game Game Game Game (18:49) Choose Your Fighter (19:48) Get My Agent On The Line (25:15) Deepfaketown and Botpocalypse Soon (29:17) Fun With Media Generation (31:10) Cyber Lack of Security (34:12) Hugging the Face (35:18) Misaligned! (37:16) A Young Lady's Illustrated Primer (38:28) They Took Our Jobs (40:45) Corporations Are Not Superintelligences (42:56) Get Involved (43:11) Introducing (45:26) In Other AI News (45:59) Show Me the Money (48:18) Quiet Speculations (50:26) Quickly, There's No Time (52:26) He's Putting Together a Team (55:14) The Quest for Sane Regulations (57:37) Well, At Least They're Forecasters (58:41) Chip City (59:16) The Week in Audio (01:03:48) Stop, Stop, He's Already Dead (01:08:32) People Just Say Things (01:10:46) Take a Moment (01:12:24) [Artificial Intelligence] (01:13:07) The American People Really Hate AI (01:16:29) Rhetorical Innovation (01:19:55) Aligning a Smarter Than Human Intelligence is Difficult (01:20:11) Open Weight Models Are Unsafe And Nothing Can Fix This (01:23:22) Cooperative Alignment (01:33:45) Building the Field (01:36:05) People Are Worried About AI Killing Everyone (01:40:03) Other People Are Not As Worried About AI Killing Everyone (01:40:38) Please Speak Directly Into This Microphone (01:42:54) The Lighter Side --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/DZPodskqJJNEkMLvH/ai-189-new-math --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  2. 23h ago

    “The Curve Bends You” by Zvi

    The plan is no plan. That is not the worst possible plan. But it is close. Since before the transformer, we have warned that the most suicidal thing you could do would be to ask your AI to do your alignment homework, and automate the process. Alignment is complex and interacts deeply with every aspect of the world, and is one of the hardest possible things for an AI to get right even if it means maximally well and is itself functionally aligned. Mistakes get amplified up the chain, you get exactly what you optimized for, and you are rather doomed. Never go full RSI. Yet, now more than ever, this seems to be the plan: Solve prosaic issues and have operational excellence, to align current AI. Have current AI do automated alignment work and figure it all out, aka ????. Profit. The good news is, most people realize this is at best a no-good terrible plan and would prefer a better one. The bad news is that some people still think this is a plan that, while not great, still has a solid chance of working, with only [...] --- Outline: (02:59) Welcome to the Chatham House (03:32) Overall Impressions (05:29) AI Is Kind Of A Big Deal, Sir (07:49) Quickly, There's No Time (10:37) The Situation is Grim (11:34) Track Trouble (15:00) AI Is Not a Normal Technology (16:00) The Plan is No Plan (19:54) The Plan is to Pace (21:45) Other Tracks (25:30) The Plan is the President (27:20) The Plan is Politics (29:39) The Plan is to Post (30:37) Are Alignment Evals Doomed? (32:24) Eternal September (34:06) Man's Search for Meaning (35:48) The Food (37:32) Chill Pill --- First published: October 7th, 2026 Source: https://www.lesswrong.com/posts/qFa5qpw9bJiQMpJks/the-curve-bends-you --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  3. 1d ago

    “Childhood and Education #21: Grades and Standards” by Zvi

    First you get to college via the wrong metrics, as previously discussed in #12. Then things get easier. I’m still digging out from under everything that happened the four days I was gone. Normal AI posting should resume tomorrow, likely with a report from the conference. Is Our Children Learning The declines in NAEP scores and other tests seem to be concentrated on the weakest students. The bottom of the distribution is falling through the floor and the rest is mostly holding up. This does not match anecdotal reports coming out of schools, which suggest widespread declines, although those are presumably untrustworthy and it's traditional to think children are in general getting dumber. That doesn’t mean they aren’t but we need systematic evidence. It's Bad, But It's Not That Bad This really would be far worse than I think if it was true. Tyler Cowen: It is worse than you think: Of 360,000 children aged 15 in Zambia only five (not 5%, 5 total) could read at “globally proficient levels.” The source is PISA. I mean, in addition to this meaning essentially college-level reading skills, it's very obviously not true. [...] --- Outline: (00:30) Is Our Children Learning (00:59) It's Bad, But It's Not That Bad (02:10) Standardized Tests Help Disadvantaged Students (04:26) Do Not Saturate Your Benchmarks (05:23) Holistic Admissions Turn Childhood Into Kayfabe (08:30) Holistic Admissions Should Mostly Be Positive Selection (09:40) Beware Stolen Valor (10:39) The Name Game (11:00) Fair Weather College (11:31) Disability Accommodations Are Now Mostly A Scam (13:19) Harvard Has Some Grade Inflation (21:06) Fighting Grade Inflation with GAAP Accounting (24:38) Not Fighting Grade Inflation (25:11) Is Our Children Learning? (26:22) Biting All The Bullets (28:00) Good Luck, Have Fun (28:53) Cheaters Gonna Cheat Cheat Cheat Cheat Cheat (35:06) Most College Students Fake Wokeness --- First published: October 6th, 2026 Source: https://www.lesswrong.com/posts/QXmyxaQnB57reu7ie/childhood-and-education-21-grades-and-standards --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  4. Oct 1

    “AI #188: Gemini Dot Argon” by Zvi

    Is Google back? They claim that they are back. Gemini 4 Argon is rolling out, with competitive frontier-level benchmarks, at 2 dollars/10 dollars. What we don’t have is access to the model, because Google Fails Marketing Forever. So it is far too early to say what we have here. When I know more, so will you. OpenAI was forced to pull what would have been GPT-6.1 Astra due to alignment failures. They did offer us GPT-6.1 Sol, which is pitched as approaching Astra quality at the much lower price of 2 dollars/10 dollars, the same as Gemini 4 Argon. The rest of OpenAI's big Dev Day announcements were Ultrafast mode and Dots, your always-on AI agent based on Astra, which comes with your Pro subscription. I’m trying it out and will report back over time if I find it useful. The new hotness remains Claude Opus 5.5. This model rocks. It has made me considerably more productive and made my day more pleasant. It should raise your ambitions. There are some particular reasons to call upon Fable 5.1 or Astra, and sometimes a cheaper model will do, but pending Argon I consider Opus 5.5 [...] --- Outline: (03:04) Language Models Offer Mundane Utility (06:12) Huh, Upgrades (07:38) Better Call Sol (11:39) Gotta Go Ultrafast (13:13) On Your Marks (16:42) Choose Your Fighter (19:16) Get My Agent On The Line (22:34) The Warner Sister (26:33) Deepfaketown and Botpocalypse Soon (30:58) Fun With Media Generation (31:53) Cyber Lack of Security (33:24) A Young Lady's Illustrated Primer (33:59) They Took Our Jobs (41:04) Levels of Friction (43:50) Get Involved (45:54) Introducing (47:54) In Other AI News (48:57) Show Me the Money (50:43) Quickly, There's No Time (52:38) Pick Up the Phone (53:36) Quest for Sane Regulations (53:58) Chip City (54:07) The Open Model Frontier Is Largely Massive Fraudulent Distillation Attacks (56:23) The Week in Audio (57:38) People Just Say Things (59:17) Rhetorical Innovation (01:03:55) Greetings From the Department of War (01:06:55) The Department of Autonomous Warfare (01:08:16) Aligning a Smarter Than Human Intelligence is Difficult (01:11:03) Cooperative Alignment (01:16:42) I'm Upping My p(doom), the Future Goes Foom (01:24:16) No, You Make a Good Point, You're Not That Persuasive (01:25:27) Muddling Through (01:27:34) The Lighter Side --- First published: October 1st, 2026 Source: https://www.lesswrong.com/posts/S2EAn9v4BwRdptsom/ai-188-gemini-dot-argon --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  5. Sep 30

    “A ‘Morally Binding’ White House Accord on AI Safety” by Zvi

    The leaders in AI were invited to the White House. We left with a White House agreement that is nonzero Actual Progress rather than a step backwards. The key to success, in many situations, is to call the whole operation something else. When you have one side that cares mostly about vibes, and the other that cares about the substance, this suggests a deal that can be struck. Suddenly everyone agrees on everything. Works for me. That doesn’t mean peace in our time. The next fight is already ramping up, as we see signs that they will make another attempt at an insane, maximally bad moratorium during the lame duck session. Table of Contents Look Who's Coming To Dinner. Let's Do Lunch. I Think It's Morally Binding, Yeah. Everyone Who is Anyone. The White House Accord on [Artificial] Intelligence. The FTC Investigates. We’re Going To Need a Stronger Regulatory Regime. [Artificial Intelligence]. Money, Dear Boy. They Are Going To Try This Moratorium Insanity Again During the Lame Duck Session. The Quest for Embedded Evaluators. Hugging the Face. Reinforcement Learning from [...] --- Outline: (00:51) Look Who's Coming To Dinner (02:37) Let's Do Lunch (04:05) I Think It's Morally Binding, Yeah (05:28) Everyone Who is Anyone (05:58) The White House Accord on [Artificial] Intelligence (11:41) The FTC Investigates (12:10) We're Going To Need a Stronger Regulatory Regime (13:30) [Artificial Intelligence] (15:20) Money, Dear Boy (18:22) They Are Going To Try This Moratorium Insanity Again During the Lame Duck Session (20:57) The Quest for Embedded Evaluators (22:17) Hugging the Face (26:12) Reinforcement Learning from Heartland Feedback (RLHF) --- First published: September 30th, 2026 Source: https://www.lesswrong.com/posts/YuqaJ5bENoyyg9eMY/a-morally-binding-white-house-accord-on-ai-safety --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  6. Sep 29

    “Astra 6.1 Pulled As Insufficiently Aligned” by Zvi

    We once again got a new set of warnings yesterday, and new movement towards living in a sane world. On the heels of its pause in inference and training due to its latest sandbox escape, OpenAI has cancelled the planned release of their next frontier model, which would have become Astra 6.1. The candidate for Astra 6.1 was found to be too misaligned, including deception and exceeding scope. This leaves Anthropic in a strong position with Opus 5.5, which means they can afford to reciprocate by holding off on Opus and Mythos level models for a bit. To add a little encouragement, the Florida Attorney General brought the fire. We’re going to need to do better. Towards that, OpenAI offered its vision of how to make a safety case for new AI model training, and they are attempting to implement it. I don’t know that it would be enough, but it would be miles ahead of where we are today if they fully implemented the real versions of all of this. There were also signs of greater cooperation across labs. A new paper came out yesterday, with authors including key people from OpenAI [...] --- Outline: (01:42) Stop, Hammertime (04:17) A Modest Proposal (04:57) Making the Safety Case (08:40) Stop In the Name of the Law (12:05) A Matter of Antitrust (14:32) Standards Authority for Frontier Models (15:27) On the Threshold Of Recursive Self-Improvement (20:06) Actual Progress --- First published: September 29th, 2026 Source: https://www.lesswrong.com/posts/gEDNSiCY2GGQrFS65/astra-6-1-pulled-as-insufficiently-aligned --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  7. Sep 28

    “What Also Happened: #NotOnlyHuggingFace” by Zvi

    OpenAI has been holding out on us. First we learned about the HuggingFace incident. They gave us a postmortem, but it was highly incomplete. Even the accompanying holy s*** METR investigation and postmortem was localized and incomplete. Then there were some other incidents involving some Wikis as message boards. Then there were some additional incidents. Then there was that time they got into Australian Medicare data. Then OpenAI dropped news on a Friday afternoon that they were making their way through a pile of various incidents and notifying the targets, but they said remarkably little in the way of new details. There was a report from a startup called Parse diving into the details of exactly how the OpenAI models pulled off parts of the HuggingFace attack, involving creating almost a million URLs and other tricks to get around the extremely narrow nature of their internet access. Then Madison Mills reported in Axios that we can raise the stakes, as OpenAI and Anthropic are collectively probing tens of thousands of security incidents. Remember Jensen Huang's ‘I know they know how to fix it’ about OpenAI from last week? Wow, did that [...] --- Outline: (02:46) Hugging Other Faces (09:48) A Wants-You-To-Know Basis (10:36) Parsing the Face (12:46) Sheepishly the Member of Technical Staff Sets the 'Days Without a Research Model Escaping its Sandbox' Sign Back to Zero (17:05) The Attempt is the First Failure (19:43) Stop, Hammertime (21:24) Whacking the Mole (23:29) Self-Replicating Prompt Injections (27:29) Levels of Friction (28:44) People Care About Private Data Violations Curiously Strongly (31:50) Alternate Universes (33:23) The Correct Response To People Still Calling This a Marketing Stunt or a Regulatory Capture Scheme (35:00) A Question of Liability (36:30) Keep Summer Safe (37:42) N Boats and Several Helicopters (39:43) Alert the Media --- First published: September 28th, 2026 Source: https://www.lesswrong.com/posts/8BL8bdeQACdgJR69Y/what-also-happened-notonlyhuggingface --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  8. Sep 27

    “The Quest for Embedded Evaluators” by Zvi

    Dario Amodei's essay We Must Pace the Frontier committed Anthropic to embedded evaluators, who would be placed inside Anthropic and given employee-level access, so they could provide outside perspective and also reports on what was happening. There is only one problem. Who will be the evaluators? OpenAI followed suit on committing to the evaluators, and also issued a milquetoast but welcome call for international coordination. I will cover that here as well. What I won’t cover today, but hope to cover tomorrow, is the latest torrent of new AI hacking incidents that came to light over the weekend, which highlights that we badly need at least embedded evaluators, and plausibly far harsher measures. For now, you need to know that there were a lot more incidents that OpenAI did not disclosed, and also a new incident at OpenAI that just happened that forced them to again pause their most advanced model. I’ll get right on sorting all that out. Table of Contents Look, All I’m Asking For Is That You Find A Highly-Qualified, Experienced, Trustworthy, Non-Conflicted Source of Embedded Evaluators That Will Work Entirely For Free, Without Government Assistance or Money from [...] --- Outline: (01:15) Look, All I'm Asking For Is That You Find A Highly-Qualified, Experienced, Trustworthy, Non-Conflicted Source of Embedded Evaluators That Will Work Entirely For Free, Without Government Assistance or Money from EA Sources Not Chosen By the Lab (04:44) Anthropic Partners with Accenture for Embedded Evaluation, also Plans to Include METR (11:04) Reading the METR (14:55) OpenAI Suggests Doing The Least We Can Do --- First published: September 27th, 2026 Source: https://www.lesswrong.com/posts/uLmf3GmBywsmG8LLZ/the-quest-for-embedded-evaluators --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

See All (250)

Ratings & Reviews

5
out of 5
2 Ratings

About

Audio narrations of LessWrong posts by zvi

Information

  • Creator
    zvi
  • Years Active
    2024 - 2026
  • Episodes
    250
  • Rating
    Explicit
  • Copyright
    © 2026 All rights reserved
  • Show Website
    LessWrong posts by zvi

You Might Also Like