Linear Digressions

Katie Malone

Demystifying AI for the intelligently curious

  1. hace 6 días

    Better Know A Benchmark: ExploitGym

    When OpenAI's frontier models were caught hacking Hugging Face's servers, most people assumed they were hunting for answer keys. The real story is stranger and more unsettling. Katie and Phoebe unpack ExploitGym — the cybersecurity benchmark at the center of the incident — and why agents are scored not just on whether they capture the flag, but on whether they used the specified vulnerability to get there. That nuance turned out to be load-bearing: the agents reverse-engineered the flags within the first hour, then spent days attacking Hugging Face to learn how the LLM judge worked so they could get their cheated answers past it. The punchline? OpenAI never had that judge switched on.

  2. 7 sep

    Constitutional AI

    How do you teach a model the difference between helpful and harmful when it has no inherent sense of either? This episode dives into Constitutional AI, Anthropic's framework for training AI systems to be both useful and safe by giving them an explicit set of principles to reason from. It's a fascinating look at how alignment research is evolving beyond simple human feedback — and what it means to give an AI something like a conscience. Links: Anthropic, "Constitutional AI: Harmlessness from AI Feedback" (2022) https://arxiv.org/abs/2212.08073 Claude's Constitution https://www.anthropic.com/constitution Anthropic, "Teaching Claude Why" (2026) https://www.anthropic.com/research/teaching-claude-why

  3. 31 ago

    A Data-Driven Reality Check on AI in Business (Interview with Tom Davenport, Babson College)

    Tom Davenport — the man who called data science "the sexiest job of the 21st century" — is back with a reality check on AI. As one of the most seasoned observers of how businesses actually adopt transformative technology, Davenport brings a rare, well-calibrated perspective to the AI hype cycle. Is this moment genuinely different from past paradigm shifts, or are we pattern-matching to a familiar story? Katie sits down with her old colleague to find out what's really happening when companies try to put AI to work.

  4. 24 ago

    Understanding AI Text Watermarking

    Anthropic just announced they're baking invisible watermarks directly into Claude's generated text — and while everyone else was busy having opinions about it, we were busy asking the more interesting question: how does it actually work? Turns out it's not hidden Unicode characters or first-letter secret codes — it's something far more elegant, operating at the level of word choice itself. We dig into Google DeepMind's SynthID text approach, published in *Nature* in 2024, to understand the clever statistical machinery behind watermarking language model outputs without anyone being the wiser.

  5. 17 ago

    Better Know a Benchmark: Humanity's Last Exam

    Humanity's Last Exam was designed with a bold premise: questions that human experts can answer, but AI models can't. Originally dubbed "Humanity's Last Stand," this benchmark is a massive academic collaboration — hundreds of contributors, thousands of fiendishly hard questions spanning a wild range of domains. In this Better Know a Benchmark installment, we unpack what HLE is actually testing, how it was built, and what it means when a model finally starts cracking it.

  6. 10 ago

    A Scientific Deep Dive into Overconfident LLMs: Interview with Kaitlyn Zhou (Cornell)

    When a language model tells you it's absolutely certain, is it actually more likely to be right? Kaitlyn Zhou's research says: not necessarily — sometimes confident phrasing correlates with *worse* accuracy, echoing a very human Dunning-Kruger effect. In this conversation, Kaitlyn (soon an assistant professor at Cornell) walks through why LLMs talk this way in the first place — tracing the tendency back through training data and the RLHF annotation process, where it turns out humans don't love confidence so much as they punish uncertainty — and what that does to the person on the other end of the chat window, who turns out to rely on confident (and even flatly-stated) answers far more than they should. We also get into her newer work on voice cloning, and how a cloned voice can sound more "native" and more trustworthy than the real one it's based on.

  7. 3 ago

    Reasoning Models: When LLMs Went Beyond Fancy Autocomplete

    Reasoning models don't just answer your question — they *think out loud* first. In this episode we dig into the class of AI models that generate intermediate chains of thought before arriving at a final answer, exploring how the internal reasoning process works. Are these models genuinely "thinking," or is something else going on under the hood?

  8. 27 jul

    Distillation, or, How to Steal a Model

    This week we’re covering model distillation: the technique of using a large "teacher" model's outputs to train a smaller, cheaper "student" model that mimics it. They cover the two big reasons labs do this — making lighter, faster, more focused models for specific tasks, and the more contentious use case of effectively copying a rival's flagship model by hammering its API with questions (with a callback to the old Bing/Google search controversy). They also get into why it's so hard to prove distillation happened, why some models occasionally introduce themselves as "Claude," and a surprisingly old idea: a 2015 paper by Geoffrey Hinton, Jeff Dean, and Oriol Vinyals on distilling knowledge using the full probability distribution over a model's outputs — not just its single most likely answer — and what that "soft label" approach captures about how a model relates concepts to each other.

Ver todo (324)

4.8
de 5
360 calificaciones

Acerca de

Demystifying AI for the intelligently curious

Información

  • Creador
    Katie Malone
  • Años de actividad
    2014 - 2026
  • Episodios
    324
  • Clasificación
    Apto
  • Copyright
    © All rights reserved
  • Mostrar sitio web
    Linear Digressions

También te podría interesar