AI Post Transformers

mcgrof
  • 3.7 (3)
  • Technology
  • Updated Daily

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

  1. 15h ago

    RaReCache: Rank-Disagreement Recomputation for Cross-Model KV Cache Reuse

    This episode explores RaReCache, a 2026 preprint on reusing a small model's KV cache inside a much larger model from the same family. It starts with the handoff cost in cascades and routers: when a cheap model escalates to a big one, the big model has to redo the full prefill. The discussion covers prior work on selective recomputation (token-level in CacheBlend, layer-level in DroidSpeak) and a closed-form ridge-regression map that translates KV entries between model sizes. The motivating result uses Qwen3-0.6B as the source and Qwen3-14B as the target on GSM8K. Plain cache mapping reaches 71.6% accuracy against 95.1% for a native prefill. The paper's argument is that the errors cluster in a few positions, so the large model only needs to recompute a fraction of tokens. The hosts also note how this repair differs from lossless speculative decoding. They flag that the calibration data comes from the same task's training split, which limits what the results show. The episode ends on why attention-based selection of positions fails: the 14B model puts most of its attention on a sink token and template text, and very little on the actual question. Sources: 1. RaReCache: Bridging the Gap in Cross-Model KV Cache Reuse via Rank disagreement-based Selective Recomputation — Sreetama Sarkar, Saptarshi Mitra, Sitao Huang, Souvik Kundu, Peter A. Beerel, 2026 http://arxiv.org/abs/2610.11358v1 2. CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion — Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang, 2024 (arXiv); EuroSys 2025 https://scholar.google.com/scholar?q=CacheBlend%3A+Fast+Large+Language+Model+Serving+for+RAG+with+Cached+Knowledge+Fusion 3. DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving — Yuhan Liu, Yuyang Huang, Jiayi Yao, Shan Lu, Junchen Jiang and collaborators, 2024 (arXiv); later versions through 2026 https://scholar.google.com/scholar?q=DroidSpeak%3A+KV+Cache+Sharing+for+Cross-LLM+Communication+and+Multi-LLM+Serving 4. Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) — Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, Ion Stoica, 2023 (SOSP) https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention+%28vLLM%29 5. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance — Lingjiao Chen, Matei Zaharia, James Zou, 2023 https://scholar.google.com/scholar?q=FrugalGPT%3A+How+to+Use+Large+Language+Models+While+Reducing+Cost+and+Improving+Performance 6. H2O: Heavy-Hitter Oracle: Efficient Generative Inference of Large Language Models with Heavy Hitters — Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, Beidi Chen, 2023 (NeurIPS) https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle%3A+Efficient+Generative+Inference+of+Large+Language+Models+with+Heavy+Hitters 7. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale — Tim Dettmers, Mike Lewis, Younes Belkada, Luke Zettlemoyer, 2022 (NeurIPS) https://scholar.google.com/scholar?q=LLM.int8%28%29%3A+8-bit+Matrix+Multiplication+for+Transformers+at+Scale 8. Palu: Compressing KV-Cache with Low-Rank Projection — Chi-Chih Chang, Wei-Cheng Lin, Chien-Yu Lin, Chong-Yan Chen, Yu-Fang Hu, Pei-Shuo Wang, Ning-Chi Huang, Luis Ceze, Mohamed S. Abdelfattah, Kai-Chiang Wu, 2024 (arXiv) https://scholar.google.com/scholar?q=Palu%3A+Compressing+KV-Cache+with+Low-Rank+Projection 9. Cross-model KV cache transfer in LLM families: A closed-form linear mapping for prefill reuse — Taekyung Heo, Rasoul Shafipour, Ritchie Zhao, et al., 2026 https://scholar.google.com/scholar?q=Cross-model+KV+cache+transfer+in+LLM+families%3A+A+closed-form+linear+mapping+for+prefill+reuse 10. Cache-to-Cache: Direct Semantic Communication Between Large Language Models — Tianyu Fu, Zihan Min, Hanling Zhang, et al., 2026 https://scholar.google.com/scholar?q=Cache-to-Cache%3A+Direct+Semantic+Communication+Between+Large+Language+Models 11. Latent Space Communication via K-V Cache Alignment — Lucio M. Dery, Zohar Yahav, Henry Prior, et al., 2026 https://scholar.google.com/scholar?q=Latent+Space+Communication+via+K-V+Cache+Alignment 12. Reduced-rank regression for the multivariate linear model — Alan Julian Izenman, 1975 https://scholar.google.com/scholar?q=Reduced-rank+regression+for+the+multivariate+linear+model 13. Efficient Streaming Language Models with Attention Sinks (StreamingLLM) — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2024 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks+%28StreamingLLM%29 14. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of LLMs; SnapKV: LLM Knows What You Are Looking for Before Generation — Zhenyu Zhang et al.; Yuhong Li et al., 2023 / 2024 https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+LLMs%3B+SnapKV%3A+LLM+Knows+What+You+Are+Looking+for+Before+Generation 15. Fast Inference from Transformers via Speculative Decoding — Yaniv Leviathan, Matan Kalman, Yossi Matias, 2023 https://scholar.google.com/scholar?q=Fast+Inference+from+Transformers+via+Speculative+Decoding 16. Palu: Compressing KV-Cache with Low-Rank Projection; Eigen Attention — Chi-Chih Chang et al.; Utkarsh Saxena et al., 2024 https://scholar.google.com/scholar?q=Palu%3A+Compressing+KV-Cache+with+Low-Rank+Projection%3B+Eigen+Attention 17. FrugalGPT / RouteLLM — Lingjiao Chen, Matei Zaharia, James Zou; Isaac Ong et al., 2023 / 2024 https://scholar.google.com/scholar?q=FrugalGPT+%2F+RouteLLM Interactive Visualization: RaReCache: Rank-Disagreement Recomputation for Cross-Model KV Cache Reuse

  2. 1d ago

    GraniKV: Asymmetric KV-Cache Paging for Shared-Prefix Agents

    This episode examines GraniKV, a KV-cache paging scheme for multi-agent workloads where many requests share a very long prompt prefix. It argues that engines like SGLang treat the shared prefix and each request's private suffix the same way, using 16-token pages, even though the two want opposite things from memory. The discussion first covers the background: PagedAttention, RadixAttention prefix reuse, and the point that reuse saves prefill compute but not the repeated memory reads during decode. It then separates two prior kernel ideas, cascade attention, which reads the prefix once to cut memory traffic, and Hydragen, which batches queries into dense matrix multiplies but needs a contiguous prefix. GraniKV's claim is about storage and integration rather than new kernels. It keeps each prefix as one contiguous slab in a reserved "hot" pool, keeps suffixes at single-token granularity in a "cold" pool, and chooses an attention backend at each decode step. The episode tests where the reported 2.16x throughput gain over stock SGLang at a 16K-token prefix comes from, asking whether the layout or the kernel does the work, with a self-built cascade baseline as the comparison. It is useful for listeners who want to understand why prefix-heavy agent serving is memory-bound and how storage layout interacts with attention kernels. Sources: 1. GraniKV: Asymmetric Granularity KV-Cache Paging for Multi-Agent Systems with Long Shared Prefix — Jinhyun Jeon, Sungjoo Yoo, 2026 http://arxiv.org/abs/2608.15584v1 2. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, Ion Stoica, 2023 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention 3. SGLang: Efficient Execution of Structured Language Model Programs — Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, Ying Sheng, 2024 https://scholar.google.com/scholar?q=SGLang%3A+Efficient+Execution+of+Structured+Language+Model+Programs 4. vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention — Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, Ashish Panwar, 2024 https://scholar.google.com/scholar?q=vAttention%3A+Dynamic+Memory+Management+for+Serving+LLMs+without+PagedAttention 5. Preble: Efficient Distributed Prompt Scheduling for LLM Serving — Vikranth Srivastava, Shu Liu, Yifan Qiao, Yang Zhou, Xingyu Xiang and colleagues (UC San Diego / UC Berkeley), 2024 https://scholar.google.com/scholar?q=Preble%3A+Efficient+Distributed+Prompt+Scheduling+for+LLM+Serving 6. Hydragen: High-Throughput LLM Inference with Shared Prefixes — Jordan Juravsky, Bradley Brown, Ryan Ehrlich, Daniel Y. Fu, Christopher Ré, Azalia Mirhoseini, 2024 https://scholar.google.com/scholar?q=Hydragen%3A+High-Throughput+LLM+Inference+with+Shared+Prefixes 7. Cascade Inference: Memory-Bandwidth Efficient Shared Prefix Batch Decoding / FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving — Zihao Ye, Ruihang Lai, Bo-Ru Lu, Chien-Yu Lin, Size Zheng, Lequn Chen, Tianqi Chen, Luis Ceze (and co-authors), 2024-2025 https://scholar.google.com/scholar?q=Cascade+Inference%3A+Memory-Bandwidth+Efficient+Shared+Prefix+Batch+Decoding+%2F+FlashInfer%3A+Efficient+and+Customizable+Attention+Engine+for+LLM+Inference+Serving 8. ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition — Lu Ye, Ze Tao, Yong Huang, Yang Li, 2024 https://scholar.google.com/scholar?q=ChunkAttention%3A+Efficient+Self-Attention+with+Prefix-Aware+KV+Cache+and+Two-Phase+Partition 9. Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs — Ben Athiwaratkun, Sujan Kumar Gonugondla, Sanjay Krishna Gouda, Haifeng Qian and colleagues (AWS), 2024 https://scholar.google.com/scholar?q=Bifurcated+Attention%3A+Accelerating+Massively+Parallel+Decoding+with+Shared+Prefixes+in+LLMs 10. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve — Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, Ramachandran Ramjee, 2024 https://scholar.google.com/scholar?q=Taming+Throughput-Latency+Tradeoff+in+LLM+Inference+with+Sarathi-Serve 11. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving — Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, Hao Zhang, 2024 https://scholar.google.com/scholar?q=DistServe%3A+Disaggregating+Prefill+and+Decoding+for+Goodput-optimized+Large+Language+Model+Serving 12. FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving — Zihao Ye, Lequn Chen, Ruihang Lai, Wuwei Lin, Yineng Zhang, Stephanie Wang, Tianqi Chen, Baris Kasikci, Vinod Grover, Arvind Krishnamurthy, Luis Ceze, 2025 https://scholar.google.com/scholar?q=FlashInfer%3A+Efficient+and+Customizable+Attention+Engine+for+LLM+Inference+Serving 13. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning / Flash-Decoding for long-context inference — Tri Dao (FlashAttention-2); Tri Dao, Daniel Haziza, Francisco Massa, Grigory Sizov (Flash-Decoding), 2023 https://scholar.google.com/scholar?q=FlashAttention-2%3A+Faster+Attention+with+Better+Parallelism+and+Work+Partitioning+%2F+Flash-Decoding+for+long-context+inference 14. FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving (including the cascade / multi-level cascade API) — Ye, Chen, Lai, Shao, Zhao, Zheng, Jin, et al., 2025 https://scholar.google.com/scholar?q=FlashInfer%3A+Efficient+and+Customizable+Attention+Engine+for+LLM+Inference+Serving+%28including+the+cascade+%2F+multi-level+cascade+API%29 15. SGLang: Efficient Execution of Structured Language Model Programs (RadixAttention) — Zheng, Yin, Xie, Sun, Huang, Yu, Cao, Kozyrakis, Stoica, Gonzalez, Barrett, Sheng, 2024 https://scholar.google.com/scholar?q=SGLang%3A+Efficient+Execution+of+Structured+Language+Model+Programs+%28RadixAttention%29 16. Efficient Memory Management for LLM Serving with PagedAttention — Kwon, Li, Zhuang, Sheng, Yu, Gonzalez, Zhang, Stoica, 2023 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+LLM+Serving+with+PagedAttention 17. HotPrefix: Hotness-aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems — Li, Gu, Huan, Wang, Yao, Tian, Chen, 2025 https://scholar.google.com/scholar?q=HotPrefix%3A+Hotness-aware+KV+Cache+Scheduling+for+Efficient+Prefix+Sharing+in+LLM+Inference+Systems 18. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving — Qin, Li, He, Zhang, Wu, Zheng, Xu, 2025 https://scholar.google.com/scholar?q=Mooncake%3A+A+KVCache-centric+Disaggregated+Architecture+for+LLM+Serving 19. FlashAttention-2 / FlashDecoding (split-KV) — Dao; Dao, Haziza, Massa, Sizov, 2023-2024 https://scholar.google.com/scholar?q=FlashAttention-2+%2F+FlashDecoding+%28split-KV%29 20. Multi-Level / Prefix-sharing attention in other engines (e.g. DeFT, Prefix-sharing tree attention) — Yao et al. (DeFT) and related tree-attention work, 2024 https://scholar.google.com/scholar?q=Multi-Level+%2F+Prefix-sharing+attention+in+other+engines+%28e.g.+DeFT%2C+Prefix-sharing+tree+attention%29 Interactive Visualization: GraniKV: Asymmetric KV-Cache Paging for Shared-Prefix Agents

  3. 1d ago

    Self-Evolving Curriculum: Bandit-Selected Problems for LLM Reasoning RL

    This episode examines Self-Evolving Curriculum, a 2025 paper proposing that a small multi-armed bandit can choose which problem categories an LLM trains on during reinforcement learning from verifiable rewards. It explains why GRPO gets no learning signal from problems the model always solves or never solves, since every advantage in the group is zero. It contrasts the bandit approach with fixed easy-to-hard schedules, random shuffling, and DAPO-style dynamic sampling, which pays for rollouts it then discards. The discussion covers the non-stationary bandit setup: each difficulty level or problem type is an arm, and the signal is reused from advantages GRPO already computes. It also looks at the Countdown, Zebra, ARC-1D and MATH experiments, including the point that the "out-of-distribution" claims are cleaner for the puzzles than for the math benchmarks. Listeners get a clear walkthrough of the mechanism and a skeptical look at whether the evidence supports the paper's claims. Sources: 1. Self-Evolving Curriculum for LLM Reasoning — Xiaoyin Chen, Jiarui Lu, Minsu Kim, Dinghuai Zhang, Jian Tang, Alexandre Piché, Nicolas Gontier, Yoshua Bengio, Ehsan Kamalloo, 2025 http://arxiv.org/abs/2505.14970 2. Curriculum Learning — Yoshua Bengio, Jérôme Louradour, Ronan Collobert, Jason Weston, 2009 https://scholar.google.com/scholar?q=Curriculum+Learning 3. Automated Curriculum Learning for Neural Networks — Alex Graves, Marc G. Bellemare, Jacob Menick, Rémi Munos, Koray Kavukcuoglu, 2017 https://scholar.google.com/scholar?q=Automated+Curriculum+Learning+for+Neural+Networks 4. Teacher-Student Curriculum Learning — Tambet Matiisen, Avital Oliver, Taco Cohen, John Schulman, 2017 (arXiv); published in IEEE Transactions on Neural Networks and Learning Systems, 2019 https://scholar.google.com/scholar?q=Teacher-Student+Curriculum+Learning 5. Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey — Sanmit Narvekar, Bei Peng, Matteo Leonetti, Jivko Sinapov, Matthew E. Taylor, Peter Stone, 2020 https://scholar.google.com/scholar?q=Curriculum+Learning+for+Reinforcement+Learning+Domains%3A+A+Framework+and+Survey 6. Efficient Online Data Mixing For Language Model Pre-Training — Alon Albalak, Liangming Pan, Colin Raffel, William Yang Wang, 2023 https://scholar.google.com/scholar?q=Efficient+Online+Data+Mixing+For+Language+Model+Pre-Training 7. DAPO: An Open-Source LLM Reinforcement Learning System at Scale — Qiying Yu et al., 2025 https://scholar.google.com/scholar?q=DAPO%3A+An+Open-Source+LLM+Reinforcement+Learning+System+at+Scale 8. DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-training — Zhenting Wang, Guofeng Cui, Kun Wan, Wentian Zhao, 2025 https://scholar.google.com/scholar?q=DUMP%3A+Automated+Distribution-Level+Curriculum+Learning+for+RL-based+LLM+Post-training 9. Efficient Reinforcement Finetuning via Adaptive Curriculum Learning (AdaRFT) — Taiwei Shi, Yiyang Wu, Linxin Song, Tianyi Zhou, Jieyu Zhao, 2025 https://scholar.google.com/scholar?q=Efficient+Reinforcement+Finetuning+via+Adaptive+Curriculum+Learning+%28AdaRFT%29 10. Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning — Sanghwan Bae et al., 2025 https://scholar.google.com/scholar?q=Online+Difficulty+Filtering+for+Reasoning+Oriented+Reinforcement+Learning 11. Prioritized Experience Replay — Tom Schaul, John Quan, Ioannis Antonoglou, David Silver, 2015 https://scholar.google.com/scholar?q=Prioritized+Experience+Replay 12. Not All Samples Are Created Equal: Deep Learning with Importance Sampling — Angelos Katharopoulos, François Fleuret, 2018 https://scholar.google.com/scholar?q=Not+All+Samples+Are+Created+Equal%3A+Deep+Learning+with+Importance+Sampling 13. Evolving Curricula with Regret-Based Environment Design (ACCEL) — Jack Parker-Holder, Minqi Jiang, Michael Dennis, et al., 2022 https://scholar.google.com/scholar?q=Evolving+Curricula+with+Regret-Based+Environment+Design+%28ACCEL%29 14. Prioritized Level Replay — Minqi Jiang, Edward Grefenstette, Tim Rocktäschel, 2021 https://scholar.google.com/scholar?q=Prioritized+Level+Replay 15. Dr. GRPO / Understanding R1-Zero-Like Training: A Critical Perspective — Zichen Liu et al., 2025 https://scholar.google.com/scholar?q=Dr.+GRPO+%2F+Understanding+R1-Zero-Like+Training%3A+A+Critical+Perspective 16. Variance-based / Prompt-difficulty selection for RLVR (e.g., Online Difficulty Filtering, GRESO, Prompt Curriculum Learning) — Various, 2025, 2025 https://scholar.google.com/scholar?q=Variance-based+%2F+Prompt-difficulty+selection+for+RLVR+%28e.g.%2C+Online+Difficulty+Filtering%2C+GRESO%2C+Prompt+Curriculum+Learning%29 Interactive Visualization: Self-Evolving Curriculum: Bandit-Selected Problems for LLM Reasoning RL

  4. 2d ago

    Attention Sinks Enable Streaming Language Models Over Endless Input

    This episode examines "Efficient Streaming Language Models with Attention Sinks" (ICLR 2024), which asks how a model trained on a fixed attention window can run over an endless input stream at constant memory and constant per-token cost, with no fine-tuning. It explains why plain window attention collapses once the first tokens are evicted from the KV cache (perplexity in the thousands), and why recomputing every window works but is slow. The key argument is that softmax forces attention to land somewhere, so models learn to park surplus attention on the initial tokens, which act as "sinks". On Llama-2-13B, keeping just the first four tokens plus a rolling window brings perplexity from about 5,158 down to 5.40, close to the 5.43 of full recomputation. Replacing those four tokens with newlines still works, which shows that absolute position matters and content doesn't. The discussion also stresses that the method delivers stable generation, not long-term memory, and it ends on how positions are assigned by cache slot rather than by original text position. Sources: 1. Efficient Streaming Language Models with Attention Sinks — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2023 http://arxiv.org/abs/2309.17453 2. Efficient Streaming Language Models with Attention Sinks — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2023 (ICLR 2024) https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks 3. Massive Activations in Large Language Models — Mingjie Sun, Xinlei Chen, J. Zico Kolter, Zhuang Liu, 2024 https://scholar.google.com/scholar?q=Massive+Activations+in+Large+Language+Models 4. Vision Transformers Need Registers — Timothée Darcet, Maxime Oquab, Julien Mairal, Piotr Bojanowski, 2023 https://scholar.google.com/scholar?q=Vision+Transformers+Need+Registers 5. Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing — Yelysei Bondarenko, Markus Nagel, Tijmen Blankevoort, 2023 https://scholar.google.com/scholar?q=Quantizable+Transformers%3A+Removing+Outliers+by+Helping+Attention+Heads+Do+Nothing 6. H2O: Heavy-Hitter Oracle: Efficient Generative Inference of Large Language Models with Heavy-Hitter Oracle for KV Cache — Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Zhangyang Wang, Beidi Chen, et al., 2023 https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle%3A+Efficient+Generative+Inference+of+Large+Language+Models+with+Heavy-Hitter+Oracle+for+KV+Cache 7. Longformer: The Long-Document Transformer — Iz Beltagy, Matthew E. Peters, Arman Cohan, 2020 https://scholar.google.com/scholar?q=Longformer%3A+The+Long-Document+Transformer 8. Efficiently Scaling Transformer Inference — Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, et al., 2022 https://scholar.google.com/scholar?q=Efficiently+Scaling+Transformer+Inference 9. Efficient Streaming Language Models with Attention Sinks (follow-up analyses): When Attention Sink Emerges in Language Models: An Empirical View — Xiangming Gu, Tianyu Pang, Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, Min Lin, 2024 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks+%28follow-up+analyses%29%3A+When+Attention+Sink+Emerges+in+Language+Models%3A+An+Empirical+View 10. Attention is Off by One (Softmax1) — Evan Miller, 2023 https://scholar.google.com/scholar?q=Attention+is+Off+by+One+%28Softmax1%29 11. LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models — Chi Han, Qifan Wang, Wenhan Xiong, Yu Chen, Heng Ji, Sinong Wang, 2023 https://scholar.google.com/scholar?q=LM-Infinite%3A+Simple+On-the-Fly+Length+Generalization+for+Large+Language+Models 12. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models — Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, Beidi Chen, 2023 https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+Large+Language+Models 13. SnapKV: LLM Knows What You are Looking for Before Generation — Yuhong Li, Yingbing Huang, Bowen Yang, et al., 2024 https://scholar.google.com/scholar?q=SnapKV%3A+LLM+Knows+What+You+are+Looking+for+Before+Generation 14. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, 2023 https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 15. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, Ion Stoica, 2023 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention 16. Mamba: Linear-Time Sequence Modeling with Selective State Spaces — Albert Gu, Tri Dao, 2023 https://scholar.google.com/scholar?q=Mamba%3A+Linear-Time+Sequence+Modeling+with+Selective+State+Spaces 17. Efficient Transformers: A Survey / Longformer — Iz Beltagy, Matthew E. Peters, Arman Cohan, 2020 https://scholar.google.com/scholar?q=Efficient+Transformers%3A+A+Survey+%2F+Longformer Interactive Visualization: Attention Sinks Enable Streaming Language Models Over Endless Input

  5. 2d ago

    In-Context Autoencoder: Compressing Long Contexts into Memory Slots

    This episode examines "In-Context Autoencoder for Context Compression in a Large Language Model" (ICLR 2024), which uses a LoRA-adapted copy of an LLM to squeeze a context into 128 learned memory slots that an unmodified, frozen decoder reads in place of the original tokens, for roughly 4x compression. It covers the two-stage training: self-supervised pretraining on the Pile with autoencoding and language-modeling objectives, then instruction tuning on a GPT-4-generated prompt-and-answer dataset. The discussion contrasts this design with Gist tokens and AutoCompressors, and with architectural fixes for long context such as sparse attention and state-space models. It also draws on the "Lost in the Middle" findings as motivation. The hosts question whether the paper's evidence supports its "long context" framing, since compressing a single passage is a different claim from handling truly long inputs. They also flag that the synthetic training data and the GPT-4 judge deserve scrutiny. Listeners get a clear explanation of how soft-prompt-style memory slots work and a critical lens on what the results do and don't show. Sources: 1. In-context Autoencoder for Context Compression in a Large Language Model — Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, Furu Wei, 2023 http://arxiv.org/abs/2307.06945 2. Learning to Compress Prompts with Gist Tokens — Jesse Mu, Xiang Lisa Li, Noah Goodman, 2023 https://scholar.google.com/scholar?q=Learning+to+Compress+Prompts+with+Gist+Tokens 3. Adapting Language Models to Compress Contexts — Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen, 2023 https://scholar.google.com/scholar?q=Adapting+Language+Models+to+Compress+Contexts 4. Compressive Transformers for Long-Range Sequence Modelling — Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Timothy P. Lillicrap, 2019 https://scholar.google.com/scholar?q=Compressive+Transformers+for+Long-Range+Sequence+Modelling 5. Perceiver: General Perception with Iterative Attention — Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, Joao Carreira, 2021 https://scholar.google.com/scholar?q=Perceiver%3A+General+Perception+with+Iterative+Attention 6. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, 2023 https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 7. Adapting Language Models to Compress Contexts (AutoCompressors) — Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen, 2023 https://scholar.google.com/scholar?q=Adapting+Language+Models+to+Compress+Contexts+%28AutoCompressors%29 8. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models — Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 2023 https://scholar.google.com/scholar?q=LLMLingua%3A+Compressing+Prompts+for+Accelerated+Inference+of+Large+Language+Models 9. Language Modeling Is Compression — Grégoire Delétang et al., 2023 https://scholar.google.com/scholar?q=Language+Modeling+Is+Compression 10. Efficient Streaming Language Models with Attention Sinks (StreamingLLM) — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2023 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks+%28StreamingLLM%29 11. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of LLMs — Zhenyu Zhang et al., 2023 https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+LLMs 12. KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache — Zirui Liu et al., 2024 https://scholar.google.com/scholar?q=KIVI%3A+A+Tuning-Free+Asymmetric+2bit+Quantization+for+KV+Cache 13. Efficient Memory Management for LLM Serving with PagedAttention (vLLM) — Woosuk Kwon et al., 2023 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+LLM+Serving+with+PagedAttention+%28vLLM%29 14. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning — Tri Dao, 2023 https://scholar.google.com/scholar?q=FlashAttention-2%3A+Faster+Attention+with+Better+Parallelism+and+Work+Partitioning 15. Learning to Compress Contexts / xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token — Xin Cheng et al., 2024 https://scholar.google.com/scholar?q=Learning+to+Compress+Contexts+%2F+xRAG%3A+Extreme+Context+Compression+for+Retrieval-augmented+Generation+with+One+Token 16. In-Context Former / 500xCompressor: Generalized Prompt Compression — Zongqian Li, Yixuan Su, Nigel Collier, 2024 https://scholar.google.com/scholar?q=In-Context+Former+%2F+500xCompressor%3A+Generalized+Prompt+Compression 17. Perceiver / Flamingo (Perceiver Resampler) and BLIP-2 (Q-Former) — Andrew Jaegle et al.; Jean-Baptiste Alayrac et al.; Junnan Li et al., 2021-2023 https://scholar.google.com/scholar?q=Perceiver+%2F+Flamingo+%28Perceiver+Resampler%29+and+BLIP-2+%28Q-Former%29 18. Prompt Cache / Cartridges-style KV compression (Learning to compress context by training a compact KV cache) — In Gim et al.; Sabri Eyuboglu et al., 2024-2025 https://scholar.google.com/scholar?q=Prompt+Cache+%2F+Cartridges-style+KV+compression+%28Learning+to+compress+context+by+training+a+compact+KV+cache%29 Interactive Visualization: In-Context Autoencoder: Compressing Long Contexts into Memory Slots

  6. 2d ago

    LLMLingua: Compressing Prompts with Small-Model Token Pruning

    This episode examines LLMLingua, a method for compressing prompts five to twenty times by using a small local language model to delete low-information tokens, so that a black-box model such as GPT-3.5 can still answer well. It explains why prompts get so long (chain-of-thought demonstrations, retrieved documents, conversation history). It also explains why quantization, pruning, and learned soft-prompt compression don't work on closed APIs. The discussion ties token surprisal and perplexity to Shannon's redundancy argument and the view of language modeling as compression. It then shows why the Selective-Context baseline fails: it scores every token independently, so a phrase repeated across demonstrations can be deleted entirely. The episode walks through the paper's components, including aligning the small model to the target and a budget controller that barely trims the instruction and question but cuts the demonstrations heavily. Along the way it flags caveats: predictable tokens aren't always unimportant, scoring is left-to-right only, and the alignment step adds under a point on GSM8K. Sources: 1. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models — Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 2023 http://arxiv.org/abs/2310.05736 2. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models — Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 2023 (EMNLP 2023) https://scholar.google.com/scholar?q=LLMLingua%3A+Compressing+Prompts+for+Accelerated+Inference+of+Large+Language+Models 3. LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression — Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 2023 (ACL 2024) https://scholar.google.com/scholar?q=LongLLMLingua%3A+Accelerating+and+Enhancing+LLMs+in+Long+Context+Scenarios+via+Prompt+Compression 4. LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression — Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, et al., 2024 (ACL Findings 2024) https://scholar.google.com/scholar?q=LLMLingua-2%3A+Data+Distillation+for+Efficient+and+Faithful+Task-Agnostic+Prompt+Compression 5. Compressing Context to Enhance Inference Efficiency of Large Language Models (Selective Context) — Yucheng Li, Bo Dong, Frank Guerin, Chenghua Lin, 2023 (EMNLP 2023) https://scholar.google.com/scholar?q=Compressing+Context+to+Enhance+Inference+Efficiency+of+Large+Language+Models+%28Selective+Context%29 6. Prediction and Entropy of Printed English — Claude E. Shannon, 1951 https://scholar.google.com/scholar?q=Prediction+and+Entropy+of+Printed+English 7. Language Models are Unsupervised Multitask Learners (GPT-2) and perplexity-based data filtering in CCNet — Radford et al. (GPT-2, 2019); Wenzek et al. (CCNet, 2020), 2019/2020 https://scholar.google.com/scholar?q=Language+Models+are+Unsupervised+Multitask+Learners+%28GPT-2%29+and+perplexity-based+data+filtering+in+CCNet 8. Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering (Selective Context) — Yucheng Li, 2023 https://scholar.google.com/scholar?q=Unlocking+Context+Constraints+of+LLMs%3A+Enhancing+Context+Efficiency+of+LLMs+with+Self-Information-Based+Content+Filtering+%28Selective+Context%29 9. Language Modeling Is Compression — Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, et al., 2023 https://scholar.google.com/scholar?q=Language+Modeling+Is+Compression 10. Learning to Compress Prompts with Gist Tokens — Jesse Mu, Xiang Lisa Li, Noah Goodman, 2023 https://scholar.google.com/scholar?q=Learning+to+Compress+Prompts+with+Gist+Tokens 11. Adapting Language Models to Compress Contexts (AutoCompressors) — Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen, 2023 https://scholar.google.com/scholar?q=Adapting+Language+Models+to+Compress+Contexts+%28AutoCompressors%29 12. In-context Autoencoder for Context Compression in a Large Language Model (ICAE) — Tao Ge, Jing Hu, Xun Wang, Si-Qing Chen, Furu Wei, 2023 https://scholar.google.com/scholar?q=In-context+Autoencoder+for+Context+Compression+in+a+Large+Language+Model+%28ICAE%29 13. Semantic Compression with Large Language Models — Henry Gilbert, Michael Sandborn, Douglas C. Schmidt, Jesse Spencer-Smith, Jules White, 2023 https://scholar.google.com/scholar?q=Semantic+Compression+with+Large+Language+Models 14. Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance — Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng, Tushar Khot, 2023 https://scholar.google.com/scholar?q=Chain-of-Thought+Hub%3A+A+Continuous+Effort+to+Measure+Large+Language+Models%27+Reasoning+Performance 15. Training Verifiers to Solve Math Word Problems (GSM8K) — Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, et al., 2021 https://scholar.google.com/scholar?q=Training+Verifiers+to+Solve+Math+Word+Problems+%28GSM8K%29 16. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, et al., 2023 https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 17. Efficient Streaming Language Models with Attention Sinks / H2O / SnapKV (KV-cache eviction line) — Xiao et al.; Zhang et al.; Li et al., 2023-2024 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks+%2F+H2O+%2F+SnapKV+%28KV-cache+eviction+line%29 Interactive Visualization: LLMLingua: Compressing Prompts with Small-Model Token Pruning

  7. 2d ago

    Speculative Prefill: Training-Free Token Importance for Faster TTFT

    This episode examines "Speculative Prefill," an ICML 2025 method that cuts time to first token by having a small model (Llama-3.1-8B) estimate which prompt tokens matter, so a large model (Llama-405B) prefills only those. It explains why prefill dominates in real traffic, where prompts are often about ten times longer than outputs. It also explains why dropping tokens differs from sparse-attention approaches like MInference, since it also removes MLP and tensor-parallel communication costs. The discussion places the work among related approaches (SnapKV, LazyLLM, GemFilter, SwiftKV, LLMLingua) as a training-free method that doesn't rely on the big model to pick tokens. It also spells out the two bets behind it: that token importance transfers between models, and that the big model tolerates a gappy prompt at a given keep rate. The episode cautions that the headline 7.66x TTFT and 7x QPS figures come from different experiments and best-case conditions. It also notes that the importance score is an attention heuristic rather than a formal information-theoretic measure. Sources: 1. Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation — Jingyu Liu, Beidi Chen, Ce Zhang, 2025 http://arxiv.org/abs/2502.02789 2. SnapKV: LLM Knows What You are Looking for Before Generation — Yuhong Li, Yingbing Huang, Bowen Yang, et al., 2024 https://scholar.google.com/scholar?q=SnapKV%3A+LLM+Knows+What+You+are+Looking+for+Before+Generation 3. LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference — Qichen Fu et al., 2024 https://scholar.google.com/scholar?q=LazyLLM%3A+Dynamic+Token+Pruning+for+Efficient+Long+Context+LLM+Inference 4. Not All Heads Matter / Quest, and RetrievalAttention-style dynamic sparse attention — Jiaming Tang et al. (Quest); Di Liu et al. (RetrievalAttention), 2024 https://scholar.google.com/scholar?q=Not+All+Heads+Matter+%2F+Quest%2C+and+RetrievalAttention-style+dynamic+sparse+attention 5. GemFilter: Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction — Zhenmei Shi, Yifei Ming, Xuan-Phi Nguyen, Yingyu Liang, Shafiq Joty, 2024 https://scholar.google.com/scholar?q=GemFilter%3A+Discovering+the+Gems+in+Early+Layers%3A+Accelerating+Long-Context+LLMs+with+1000x+Input+Token+Reduction 6. SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation — Aurick Qiao et al. (Snowflake), 2024 https://scholar.google.com/scholar?q=SwiftKV%3A+Fast+Prefill-Optimized+Inference+with+Knowledge-Preserving+Model+Transformation 7. Fast Inference from Transformers via Speculative Decoding — Yaniv Leviathan, Matan Kalman, Yossi Matias, 2023 https://scholar.google.com/scholar?q=Fast+Inference+from+Transformers+via+Speculative+Decoding 8. LLMLingua-2 / LongLLMLingua: Question-aware prompt compression — Huiqiang Jiang, Qianhui Wu, et al., 2023-2024 https://scholar.google.com/scholar?q=LLMLingua-2+%2F+LongLLMLingua%3A+Question-aware+prompt+compression 9. Sarathi-Serve / Chunked prefill and Mooncake / prefill-decode disaggregation — Agrawal et al. (Sarathi-Serve); Qin et al. (Mooncake), 2024 https://scholar.google.com/scholar?q=Sarathi-Serve+%2F+Chunked+prefill+and+Mooncake+%2F+prefill-decode+disaggregation 10. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu et al., 2023 https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 11. Marconi / prefix-caching for hybrid LLMs and LMCache/CacheBlend — Pan et al. (Marconi); Yao et al. (CacheBlend), 2024-2025 https://scholar.google.com/scholar?q=Marconi+%2F+prefix-caching+for+hybrid+LLMs+and+LMCache%2FCacheBlend Interactive Visualization: Speculative Prefill: Training-Free Token Importance for Faster TTFT

  8. 3d ago

    AI R&D Automation and the Intelligence Explosion Question

    This episode examines a policy paper asking what happens if AI automates AI research itself, setting off an "intelligence explosion" that compresses years of progress into months. It reviews the evidence that automation is already underway, including the AI share of approved code at one lab rising from low single digits to over 80 percent, autonomous R&D work growing from 1 to 26 percent, and task-length horizons doubling about every three months. It also covers the caveats on that evidence, since coding benchmarks don't necessarily translate to research ability. The hosts explain the paper's feedback-loop model, in which a ratio of labor returns to idea-finding difficulty decides whether progress fades, holds steady or accelerates. They also cover the four frictions that could stop the loop (diminishing returns, compute limits, hard-to-automate research taste, and slow training runs) and the stakes of loss of control and eroded checks on power. Listeners get a clear walkthrough of what the argument claims and what it doesn't: it argues for preparation, not that an explosion is certain. Sources: 1. What if automating AI R&D triggers an intelligence explosion? — Alan Chan, Christoph Winter, Andrew Barto, Jakub Pachocki, Geoffrey Hinton, Eric Horvitz, Yoshua Bengio, Dawn Song, Jack Clark, Hilary Greaves, Anton Korinek, Samuel Hammond, Thore Graepel, Ben Bariach, Philip H. S. Torr, Sheila A. McIlraith, Jeff Clune, Sam Manning, Girish Sastry, Tom Davidson, Daniel Eth, Sören Mindermann, 2026 http://arxiv.org/abs/2609.36054 2. Speculations Concerning the First Ultraintelligent Machine — I.J. Good, 1965 https://scholar.google.com/scholar?q=Speculations+Concerning+the+First+Ultraintelligent+Machine 3. Intelligence Explosion Microeconomics — Eliezer Yudkowsky, 2013 https://scholar.google.com/scholar?q=Intelligence+Explosion+Microeconomics 4. Are Ideas Getting Harder to Find? — Nicholas Bloom, Charles I. Jones, John Van Reenen, Michael Webb, 2020 https://scholar.google.com/scholar?q=Are+Ideas+Getting+Harder+to+Find%3F 5. Algorithmic progress in language models — Anson Ho, Tamay Besiroglu, Ege Erdil, David Owen, Robi Rahman, Zifan Carson Guo, David Atkinson, Neil Thompson, Jaime Sevilla (Epoch AI), 2024 https://scholar.google.com/scholar?q=Algorithmic+progress+in+language+models 6. RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts — Hjalmar Wijk, Tao Lin, Joel Becker, et al. (METR), 2024 https://scholar.google.com/scholar?q=RE-Bench%3A+Evaluating+frontier+AI+R%26D+capabilities+of+language+model+agents+against+human+experts 7. Measuring AI Ability to Complete Long Tasks — Thomas Kwa, Ben West, Joel Becker, et al. (METR), 2025 https://scholar.google.com/scholar?q=Measuring+AI+Ability+to+Complete+Long+Tasks 8. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery — Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, David Ha, 2024 https://scholar.google.com/scholar?q=The+AI+Scientist%3A+Towards+Fully+Automated+Open-Ended+Scientific+Discovery 9. AI 2027 — Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, Romeo Dean, 2025 https://scholar.google.com/scholar?q=AI+2027 10. The Alignment Problem from a Deep Learning Perspective — Richard Ngo, Lawrence Chan, Sören Mindermann, 2022 https://scholar.google.com/scholar?q=The+Alignment+Problem+from+a+Deep+Learning+Perspective 11. AI Control: Improving Safety Despite Intentional Subversion — Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan, Fabien Roger, 2023 https://scholar.google.com/scholar?q=AI+Control%3A+Improving+Safety+Despite+Intentional+Subversion 12. Alignment faking in large language models — Ryan Greenblatt, Carson Denison, Benjamin Wright, et al. (Anthropic and Redwood Research), 2024 https://scholar.google.com/scholar?q=Alignment+faking+in+large+language+models 13. Managing extreme AI risks amid rapid progress — Yoshua Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, et al., 2024 https://scholar.google.com/scholar?q=Managing+extreme+AI+risks+amid+rapid+progress 14. Will compute bottlenecks prevent an intelligence explosion? — Parker Whitfill, Cheryl Wu, 2025 https://scholar.google.com/scholar?q=Will+compute+bottlenecks+prevent+an+intelligence+explosion%3F 15. The software intelligence explosion debate needs experiments — Anson Ho, Parker Whitfill, 2025 https://scholar.google.com/scholar?q=The+software+intelligence+explosion+debate+needs+experiments 16. Will AI R&D Automation Cause a Software Intelligence Explosion? — Daniel Eth, Tom Davidson, 2025 https://scholar.google.com/scholar?q=Will+AI+R%26D+Automation+Cause+a+Software+Intelligence+Explosion%3F 17. Measuring AI Ability to Complete Long Tasks / Time Horizon 1.1 — METR, 2025/2026 https://scholar.google.com/scholar?q=Measuring+AI+Ability+to+Complete+Long+Tasks+%2F+Time+Horizon+1.1 18. When does automating AI research produce explosive growth? Feedback loops in innovation networks — Tom Davidson, Basil Halperin, Thomas Houlden, Anton Korinek, 2026 https://scholar.google.com/scholar?q=When+does+automating+AI+research+produce+explosive+growth%3F+Feedback+loops+in+innovation+networks 19. The economics of recursive self-improvement — Tom Cunningham et al., 2026 https://scholar.google.com/scholar?q=The+economics+of+recursive+self-improvement 20. The bounded parallelizability of R&D: Theory and application to AI — Philip Trammell, 2026 https://scholar.google.com/scholar?q=The+bounded+parallelizability+of+R%26D%3A+Theory+and+application+to+AI 21. On the origin of algorithmic progress in AI — Hans Gundlach et al., 2025 https://scholar.google.com/scholar?q=On+the+origin+of+algorithmic+progress+in+AI 22. The dynamics of intelligence explosions — Toby Ord, 2026 https://scholar.google.com/scholar?q=The+dynamics+of+intelligence+explosions 23. PostTrainBench: Can LLM agents automate LLM post-training? — Rank et al., 2026 https://scholar.google.com/scholar?q=PostTrainBench%3A+Can+LLM+agents+automate+LLM+post-training%3F 24. Towards end-to-end automation of AI research (AI Scientist) — Chris Lu, Cong Lu, Robert Lange, Jeff Clune et al., 2026 https://scholar.google.com/scholar?q=Towards+end-to-end+automation+of+AI+research+%28AI+Scientist%29 25. Frontier AI auditing: Toward rigorous third-party assessment of safety and security practices at leading AI companies — Miles Brundage et al., 2026 https://scholar.google.com/scholar?q=Frontier+AI+auditing%3A+Toward+rigorous+third-party+assessment+of+safety+and+security+practices+at+leading+AI+companies Interactive Visualization: AI R&D Automation and the Intelligence Explosion Question

See All (972)

Ratings & Reviews

3.7
out of 5
3 Ratings

About

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

Information

You Might Also Like