AI Post Transformers

mcgrof
  • 3.7 (3)
  • Technology
  • Updated Daily

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

  1. 1d ago

    Attention Sinks Enable Streaming Language Models Over Endless Input

    This episode examines "Efficient Streaming Language Models with Attention Sinks" (ICLR 2024), which asks how a model trained on a fixed attention window can run over an endless input stream at constant memory and constant per-token cost, with no fine-tuning. It explains why plain window attention collapses once the first tokens are evicted from the KV cache (perplexity in the thousands), and why recomputing every window works but is slow. The key argument is that softmax forces attention to land somewhere, so models learn to park surplus attention on the initial tokens, which act as "sinks". On Llama-2-13B, keeping just the first four tokens plus a rolling window brings perplexity from about 5,158 down to 5.40, close to the 5.43 of full recomputation. Replacing those four tokens with newlines still works, which shows that absolute position matters and content doesn't. The discussion also stresses that the method delivers stable generation, not long-term memory, and it ends on how positions are assigned by cache slot rather than by original text position. Sources: 1. Efficient Streaming Language Models with Attention Sinks — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2023 http://arxiv.org/abs/2309.17453 2. Efficient Streaming Language Models with Attention Sinks — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2023 (ICLR 2024) https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks 3. Massive Activations in Large Language Models — Mingjie Sun, Xinlei Chen, J. Zico Kolter, Zhuang Liu, 2024 https://scholar.google.com/scholar?q=Massive+Activations+in+Large+Language+Models 4. Vision Transformers Need Registers — Timothée Darcet, Maxime Oquab, Julien Mairal, Piotr Bojanowski, 2023 https://scholar.google.com/scholar?q=Vision+Transformers+Need+Registers 5. Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing — Yelysei Bondarenko, Markus Nagel, Tijmen Blankevoort, 2023 https://scholar.google.com/scholar?q=Quantizable+Transformers%3A+Removing+Outliers+by+Helping+Attention+Heads+Do+Nothing 6. H2O: Heavy-Hitter Oracle: Efficient Generative Inference of Large Language Models with Heavy-Hitter Oracle for KV Cache — Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Zhangyang Wang, Beidi Chen, et al., 2023 https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle%3A+Efficient+Generative+Inference+of+Large+Language+Models+with+Heavy-Hitter+Oracle+for+KV+Cache 7. Longformer: The Long-Document Transformer — Iz Beltagy, Matthew E. Peters, Arman Cohan, 2020 https://scholar.google.com/scholar?q=Longformer%3A+The+Long-Document+Transformer 8. Efficiently Scaling Transformer Inference — Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, et al., 2022 https://scholar.google.com/scholar?q=Efficiently+Scaling+Transformer+Inference 9. Efficient Streaming Language Models with Attention Sinks (follow-up analyses): When Attention Sink Emerges in Language Models: An Empirical View — Xiangming Gu, Tianyu Pang, Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, Min Lin, 2024 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks+%28follow-up+analyses%29%3A+When+Attention+Sink+Emerges+in+Language+Models%3A+An+Empirical+View 10. Attention is Off by One (Softmax1) — Evan Miller, 2023 https://scholar.google.com/scholar?q=Attention+is+Off+by+One+%28Softmax1%29 11. LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models — Chi Han, Qifan Wang, Wenhan Xiong, Yu Chen, Heng Ji, Sinong Wang, 2023 https://scholar.google.com/scholar?q=LM-Infinite%3A+Simple+On-the-Fly+Length+Generalization+for+Large+Language+Models 12. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models — Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, Beidi Chen, 2023 https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+Large+Language+Models 13. SnapKV: LLM Knows What You are Looking for Before Generation — Yuhong Li, Yingbing Huang, Bowen Yang, et al., 2024 https://scholar.google.com/scholar?q=SnapKV%3A+LLM+Knows+What+You+are+Looking+for+Before+Generation 14. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, 2023 https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 15. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, Ion Stoica, 2023 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention 16. Mamba: Linear-Time Sequence Modeling with Selective State Spaces — Albert Gu, Tri Dao, 2023 https://scholar.google.com/scholar?q=Mamba%3A+Linear-Time+Sequence+Modeling+with+Selective+State+Spaces 17. Efficient Transformers: A Survey / Longformer — Iz Beltagy, Matthew E. Peters, Arman Cohan, 2020 https://scholar.google.com/scholar?q=Efficient+Transformers%3A+A+Survey+%2F+Longformer Interactive Visualization: Attention Sinks Enable Streaming Language Models Over Endless Input

  2. 1d ago

    In-Context Autoencoder: Compressing Long Contexts into Memory Slots

    This episode examines "In-Context Autoencoder for Context Compression in a Large Language Model" (ICLR 2024), which uses a LoRA-adapted copy of an LLM to squeeze a context into 128 learned memory slots that an unmodified, frozen decoder reads in place of the original tokens, for roughly 4x compression. It covers the two-stage training: self-supervised pretraining on the Pile with autoencoding and language-modeling objectives, then instruction tuning on a GPT-4-generated prompt-and-answer dataset. The discussion contrasts this design with Gist tokens and AutoCompressors, and with architectural fixes for long context such as sparse attention and state-space models. It also draws on the "Lost in the Middle" findings as motivation. The hosts question whether the paper's evidence supports its "long context" framing, since compressing a single passage is a different claim from handling truly long inputs. They also flag that the synthetic training data and the GPT-4 judge deserve scrutiny. Listeners get a clear explanation of how soft-prompt-style memory slots work and a critical lens on what the results do and don't show. Sources: 1. In-context Autoencoder for Context Compression in a Large Language Model — Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, Furu Wei, 2023 http://arxiv.org/abs/2307.06945 2. Learning to Compress Prompts with Gist Tokens — Jesse Mu, Xiang Lisa Li, Noah Goodman, 2023 https://scholar.google.com/scholar?q=Learning+to+Compress+Prompts+with+Gist+Tokens 3. Adapting Language Models to Compress Contexts — Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen, 2023 https://scholar.google.com/scholar?q=Adapting+Language+Models+to+Compress+Contexts 4. Compressive Transformers for Long-Range Sequence Modelling — Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Timothy P. Lillicrap, 2019 https://scholar.google.com/scholar?q=Compressive+Transformers+for+Long-Range+Sequence+Modelling 5. Perceiver: General Perception with Iterative Attention — Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, Joao Carreira, 2021 https://scholar.google.com/scholar?q=Perceiver%3A+General+Perception+with+Iterative+Attention 6. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, 2023 https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 7. Adapting Language Models to Compress Contexts (AutoCompressors) — Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen, 2023 https://scholar.google.com/scholar?q=Adapting+Language+Models+to+Compress+Contexts+%28AutoCompressors%29 8. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models — Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 2023 https://scholar.google.com/scholar?q=LLMLingua%3A+Compressing+Prompts+for+Accelerated+Inference+of+Large+Language+Models 9. Language Modeling Is Compression — Grégoire Delétang et al., 2023 https://scholar.google.com/scholar?q=Language+Modeling+Is+Compression 10. Efficient Streaming Language Models with Attention Sinks (StreamingLLM) — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2023 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks+%28StreamingLLM%29 11. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of LLMs — Zhenyu Zhang et al., 2023 https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+LLMs 12. KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache — Zirui Liu et al., 2024 https://scholar.google.com/scholar?q=KIVI%3A+A+Tuning-Free+Asymmetric+2bit+Quantization+for+KV+Cache 13. Efficient Memory Management for LLM Serving with PagedAttention (vLLM) — Woosuk Kwon et al., 2023 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+LLM+Serving+with+PagedAttention+%28vLLM%29 14. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning — Tri Dao, 2023 https://scholar.google.com/scholar?q=FlashAttention-2%3A+Faster+Attention+with+Better+Parallelism+and+Work+Partitioning 15. Learning to Compress Contexts / xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token — Xin Cheng et al., 2024 https://scholar.google.com/scholar?q=Learning+to+Compress+Contexts+%2F+xRAG%3A+Extreme+Context+Compression+for+Retrieval-augmented+Generation+with+One+Token 16. In-Context Former / 500xCompressor: Generalized Prompt Compression — Zongqian Li, Yixuan Su, Nigel Collier, 2024 https://scholar.google.com/scholar?q=In-Context+Former+%2F+500xCompressor%3A+Generalized+Prompt+Compression 17. Perceiver / Flamingo (Perceiver Resampler) and BLIP-2 (Q-Former) — Andrew Jaegle et al.; Jean-Baptiste Alayrac et al.; Junnan Li et al., 2021-2023 https://scholar.google.com/scholar?q=Perceiver+%2F+Flamingo+%28Perceiver+Resampler%29+and+BLIP-2+%28Q-Former%29 18. Prompt Cache / Cartridges-style KV compression (Learning to compress context by training a compact KV cache) — In Gim et al.; Sabri Eyuboglu et al., 2024-2025 https://scholar.google.com/scholar?q=Prompt+Cache+%2F+Cartridges-style+KV+compression+%28Learning+to+compress+context+by+training+a+compact+KV+cache%29 Interactive Visualization: In-Context Autoencoder: Compressing Long Contexts into Memory Slots

  3. 1d ago

    LLMLingua: Compressing Prompts with Small-Model Token Pruning

    This episode examines LLMLingua, a method for compressing prompts five to twenty times by using a small local language model to delete low-information tokens, so that a black-box model such as GPT-3.5 can still answer well. It explains why prompts get so long (chain-of-thought demonstrations, retrieved documents, conversation history). It also explains why quantization, pruning, and learned soft-prompt compression don't work on closed APIs. The discussion ties token surprisal and perplexity to Shannon's redundancy argument and the view of language modeling as compression. It then shows why the Selective-Context baseline fails: it scores every token independently, so a phrase repeated across demonstrations can be deleted entirely. The episode walks through the paper's components, including aligning the small model to the target and a budget controller that barely trims the instruction and question but cuts the demonstrations heavily. Along the way it flags caveats: predictable tokens aren't always unimportant, scoring is left-to-right only, and the alignment step adds under a point on GSM8K. Sources: 1. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models — Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 2023 http://arxiv.org/abs/2310.05736 2. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models — Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 2023 (EMNLP 2023) https://scholar.google.com/scholar?q=LLMLingua%3A+Compressing+Prompts+for+Accelerated+Inference+of+Large+Language+Models 3. LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression — Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu, 2023 (ACL 2024) https://scholar.google.com/scholar?q=LongLLMLingua%3A+Accelerating+and+Enhancing+LLMs+in+Long+Context+Scenarios+via+Prompt+Compression 4. LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression — Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, et al., 2024 (ACL Findings 2024) https://scholar.google.com/scholar?q=LLMLingua-2%3A+Data+Distillation+for+Efficient+and+Faithful+Task-Agnostic+Prompt+Compression 5. Compressing Context to Enhance Inference Efficiency of Large Language Models (Selective Context) — Yucheng Li, Bo Dong, Frank Guerin, Chenghua Lin, 2023 (EMNLP 2023) https://scholar.google.com/scholar?q=Compressing+Context+to+Enhance+Inference+Efficiency+of+Large+Language+Models+%28Selective+Context%29 6. Prediction and Entropy of Printed English — Claude E. Shannon, 1951 https://scholar.google.com/scholar?q=Prediction+and+Entropy+of+Printed+English 7. Language Models are Unsupervised Multitask Learners (GPT-2) and perplexity-based data filtering in CCNet — Radford et al. (GPT-2, 2019); Wenzek et al. (CCNet, 2020), 2019/2020 https://scholar.google.com/scholar?q=Language+Models+are+Unsupervised+Multitask+Learners+%28GPT-2%29+and+perplexity-based+data+filtering+in+CCNet 8. Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering (Selective Context) — Yucheng Li, 2023 https://scholar.google.com/scholar?q=Unlocking+Context+Constraints+of+LLMs%3A+Enhancing+Context+Efficiency+of+LLMs+with+Self-Information-Based+Content+Filtering+%28Selective+Context%29 9. Language Modeling Is Compression — Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, et al., 2023 https://scholar.google.com/scholar?q=Language+Modeling+Is+Compression 10. Learning to Compress Prompts with Gist Tokens — Jesse Mu, Xiang Lisa Li, Noah Goodman, 2023 https://scholar.google.com/scholar?q=Learning+to+Compress+Prompts+with+Gist+Tokens 11. Adapting Language Models to Compress Contexts (AutoCompressors) — Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen, 2023 https://scholar.google.com/scholar?q=Adapting+Language+Models+to+Compress+Contexts+%28AutoCompressors%29 12. In-context Autoencoder for Context Compression in a Large Language Model (ICAE) — Tao Ge, Jing Hu, Xun Wang, Si-Qing Chen, Furu Wei, 2023 https://scholar.google.com/scholar?q=In-context+Autoencoder+for+Context+Compression+in+a+Large+Language+Model+%28ICAE%29 13. Semantic Compression with Large Language Models — Henry Gilbert, Michael Sandborn, Douglas C. Schmidt, Jesse Spencer-Smith, Jules White, 2023 https://scholar.google.com/scholar?q=Semantic+Compression+with+Large+Language+Models 14. Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance — Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng, Tushar Khot, 2023 https://scholar.google.com/scholar?q=Chain-of-Thought+Hub%3A+A+Continuous+Effort+to+Measure+Large+Language+Models%27+Reasoning+Performance 15. Training Verifiers to Solve Math Word Problems (GSM8K) — Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, et al., 2021 https://scholar.google.com/scholar?q=Training+Verifiers+to+Solve+Math+Word+Problems+%28GSM8K%29 16. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, et al., 2023 https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 17. Efficient Streaming Language Models with Attention Sinks / H2O / SnapKV (KV-cache eviction line) — Xiao et al.; Zhang et al.; Li et al., 2023-2024 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks+%2F+H2O+%2F+SnapKV+%28KV-cache+eviction+line%29 Interactive Visualization: LLMLingua: Compressing Prompts with Small-Model Token Pruning

  4. 1d ago

    Speculative Prefill: Training-Free Token Importance for Faster TTFT

    This episode examines "Speculative Prefill," an ICML 2025 method that cuts time to first token by having a small model (Llama-3.1-8B) estimate which prompt tokens matter, so a large model (Llama-405B) prefills only those. It explains why prefill dominates in real traffic, where prompts are often about ten times longer than outputs. It also explains why dropping tokens differs from sparse-attention approaches like MInference, since it also removes MLP and tensor-parallel communication costs. The discussion places the work among related approaches (SnapKV, LazyLLM, GemFilter, SwiftKV, LLMLingua) as a training-free method that doesn't rely on the big model to pick tokens. It also spells out the two bets behind it: that token importance transfers between models, and that the big model tolerates a gappy prompt at a given keep rate. The episode cautions that the headline 7.66x TTFT and 7x QPS figures come from different experiments and best-case conditions. It also notes that the importance score is an attention heuristic rather than a formal information-theoretic measure. Sources: 1. Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation — Jingyu Liu, Beidi Chen, Ce Zhang, 2025 http://arxiv.org/abs/2502.02789 2. SnapKV: LLM Knows What You are Looking for Before Generation — Yuhong Li, Yingbing Huang, Bowen Yang, et al., 2024 https://scholar.google.com/scholar?q=SnapKV%3A+LLM+Knows+What+You+are+Looking+for+Before+Generation 3. LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference — Qichen Fu et al., 2024 https://scholar.google.com/scholar?q=LazyLLM%3A+Dynamic+Token+Pruning+for+Efficient+Long+Context+LLM+Inference 4. Not All Heads Matter / Quest, and RetrievalAttention-style dynamic sparse attention — Jiaming Tang et al. (Quest); Di Liu et al. (RetrievalAttention), 2024 https://scholar.google.com/scholar?q=Not+All+Heads+Matter+%2F+Quest%2C+and+RetrievalAttention-style+dynamic+sparse+attention 5. GemFilter: Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction — Zhenmei Shi, Yifei Ming, Xuan-Phi Nguyen, Yingyu Liang, Shafiq Joty, 2024 https://scholar.google.com/scholar?q=GemFilter%3A+Discovering+the+Gems+in+Early+Layers%3A+Accelerating+Long-Context+LLMs+with+1000x+Input+Token+Reduction 6. SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation — Aurick Qiao et al. (Snowflake), 2024 https://scholar.google.com/scholar?q=SwiftKV%3A+Fast+Prefill-Optimized+Inference+with+Knowledge-Preserving+Model+Transformation 7. Fast Inference from Transformers via Speculative Decoding — Yaniv Leviathan, Matan Kalman, Yossi Matias, 2023 https://scholar.google.com/scholar?q=Fast+Inference+from+Transformers+via+Speculative+Decoding 8. LLMLingua-2 / LongLLMLingua: Question-aware prompt compression — Huiqiang Jiang, Qianhui Wu, et al., 2023-2024 https://scholar.google.com/scholar?q=LLMLingua-2+%2F+LongLLMLingua%3A+Question-aware+prompt+compression 9. Sarathi-Serve / Chunked prefill and Mooncake / prefill-decode disaggregation — Agrawal et al. (Sarathi-Serve); Qin et al. (Mooncake), 2024 https://scholar.google.com/scholar?q=Sarathi-Serve+%2F+Chunked+prefill+and+Mooncake+%2F+prefill-decode+disaggregation 10. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu et al., 2023 https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 11. Marconi / prefix-caching for hybrid LLMs and LMCache/CacheBlend — Pan et al. (Marconi); Yao et al. (CacheBlend), 2024-2025 https://scholar.google.com/scholar?q=Marconi+%2F+prefix-caching+for+hybrid+LLMs+and+LMCache%2FCacheBlend Interactive Visualization: Speculative Prefill: Training-Free Token Importance for Faster TTFT

  5. 2d ago

    AI R&D Automation and the Intelligence Explosion Question

    This episode examines a policy paper asking what happens if AI automates AI research itself, setting off an "intelligence explosion" that compresses years of progress into months. It reviews the evidence that automation is already underway, including the AI share of approved code at one lab rising from low single digits to over 80 percent, autonomous R&D work growing from 1 to 26 percent, and task-length horizons doubling about every three months. It also covers the caveats on that evidence, since coding benchmarks don't necessarily translate to research ability. The hosts explain the paper's feedback-loop model, in which a ratio of labor returns to idea-finding difficulty decides whether progress fades, holds steady or accelerates. They also cover the four frictions that could stop the loop (diminishing returns, compute limits, hard-to-automate research taste, and slow training runs) and the stakes of loss of control and eroded checks on power. Listeners get a clear walkthrough of what the argument claims and what it doesn't: it argues for preparation, not that an explosion is certain. Sources: 1. What if automating AI R&D triggers an intelligence explosion? — Alan Chan, Christoph Winter, Andrew Barto, Jakub Pachocki, Geoffrey Hinton, Eric Horvitz, Yoshua Bengio, Dawn Song, Jack Clark, Hilary Greaves, Anton Korinek, Samuel Hammond, Thore Graepel, Ben Bariach, Philip H. S. Torr, Sheila A. McIlraith, Jeff Clune, Sam Manning, Girish Sastry, Tom Davidson, Daniel Eth, Sören Mindermann, 2026 http://arxiv.org/abs/2609.36054 2. Speculations Concerning the First Ultraintelligent Machine — I.J. Good, 1965 https://scholar.google.com/scholar?q=Speculations+Concerning+the+First+Ultraintelligent+Machine 3. Intelligence Explosion Microeconomics — Eliezer Yudkowsky, 2013 https://scholar.google.com/scholar?q=Intelligence+Explosion+Microeconomics 4. Are Ideas Getting Harder to Find? — Nicholas Bloom, Charles I. Jones, John Van Reenen, Michael Webb, 2020 https://scholar.google.com/scholar?q=Are+Ideas+Getting+Harder+to+Find%3F 5. Algorithmic progress in language models — Anson Ho, Tamay Besiroglu, Ege Erdil, David Owen, Robi Rahman, Zifan Carson Guo, David Atkinson, Neil Thompson, Jaime Sevilla (Epoch AI), 2024 https://scholar.google.com/scholar?q=Algorithmic+progress+in+language+models 6. RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts — Hjalmar Wijk, Tao Lin, Joel Becker, et al. (METR), 2024 https://scholar.google.com/scholar?q=RE-Bench%3A+Evaluating+frontier+AI+R%26D+capabilities+of+language+model+agents+against+human+experts 7. Measuring AI Ability to Complete Long Tasks — Thomas Kwa, Ben West, Joel Becker, et al. (METR), 2025 https://scholar.google.com/scholar?q=Measuring+AI+Ability+to+Complete+Long+Tasks 8. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery — Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, David Ha, 2024 https://scholar.google.com/scholar?q=The+AI+Scientist%3A+Towards+Fully+Automated+Open-Ended+Scientific+Discovery 9. AI 2027 — Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, Romeo Dean, 2025 https://scholar.google.com/scholar?q=AI+2027 10. The Alignment Problem from a Deep Learning Perspective — Richard Ngo, Lawrence Chan, Sören Mindermann, 2022 https://scholar.google.com/scholar?q=The+Alignment+Problem+from+a+Deep+Learning+Perspective 11. AI Control: Improving Safety Despite Intentional Subversion — Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan, Fabien Roger, 2023 https://scholar.google.com/scholar?q=AI+Control%3A+Improving+Safety+Despite+Intentional+Subversion 12. Alignment faking in large language models — Ryan Greenblatt, Carson Denison, Benjamin Wright, et al. (Anthropic and Redwood Research), 2024 https://scholar.google.com/scholar?q=Alignment+faking+in+large+language+models 13. Managing extreme AI risks amid rapid progress — Yoshua Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, et al., 2024 https://scholar.google.com/scholar?q=Managing+extreme+AI+risks+amid+rapid+progress 14. Will compute bottlenecks prevent an intelligence explosion? — Parker Whitfill, Cheryl Wu, 2025 https://scholar.google.com/scholar?q=Will+compute+bottlenecks+prevent+an+intelligence+explosion%3F 15. The software intelligence explosion debate needs experiments — Anson Ho, Parker Whitfill, 2025 https://scholar.google.com/scholar?q=The+software+intelligence+explosion+debate+needs+experiments 16. Will AI R&D Automation Cause a Software Intelligence Explosion? — Daniel Eth, Tom Davidson, 2025 https://scholar.google.com/scholar?q=Will+AI+R%26D+Automation+Cause+a+Software+Intelligence+Explosion%3F 17. Measuring AI Ability to Complete Long Tasks / Time Horizon 1.1 — METR, 2025/2026 https://scholar.google.com/scholar?q=Measuring+AI+Ability+to+Complete+Long+Tasks+%2F+Time+Horizon+1.1 18. When does automating AI research produce explosive growth? Feedback loops in innovation networks — Tom Davidson, Basil Halperin, Thomas Houlden, Anton Korinek, 2026 https://scholar.google.com/scholar?q=When+does+automating+AI+research+produce+explosive+growth%3F+Feedback+loops+in+innovation+networks 19. The economics of recursive self-improvement — Tom Cunningham et al., 2026 https://scholar.google.com/scholar?q=The+economics+of+recursive+self-improvement 20. The bounded parallelizability of R&D: Theory and application to AI — Philip Trammell, 2026 https://scholar.google.com/scholar?q=The+bounded+parallelizability+of+R%26D%3A+Theory+and+application+to+AI 21. On the origin of algorithmic progress in AI — Hans Gundlach et al., 2025 https://scholar.google.com/scholar?q=On+the+origin+of+algorithmic+progress+in+AI 22. The dynamics of intelligence explosions — Toby Ord, 2026 https://scholar.google.com/scholar?q=The+dynamics+of+intelligence+explosions 23. PostTrainBench: Can LLM agents automate LLM post-training? — Rank et al., 2026 https://scholar.google.com/scholar?q=PostTrainBench%3A+Can+LLM+agents+automate+LLM+post-training%3F 24. Towards end-to-end automation of AI research (AI Scientist) — Chris Lu, Cong Lu, Robert Lange, Jeff Clune et al., 2026 https://scholar.google.com/scholar?q=Towards+end-to-end+automation+of+AI+research+%28AI+Scientist%29 25. Frontier AI auditing: Toward rigorous third-party assessment of safety and security practices at leading AI companies — Miles Brundage et al., 2026 https://scholar.google.com/scholar?q=Frontier+AI+auditing%3A+Toward+rigorous+third-party+assessment+of+safety+and+security+practices+at+leading+AI+companies Interactive Visualization: AI R&D Automation and the Intelligence Explosion Question

  6. 2d ago

    ATLAS: Full-Stack Simulator for 3D-DRAM LLM Accelerators

    This episode examines ATLAS, a simulator for 3D-DRAM-based LLM accelerators that the authors say was checked against real silicon. It explains why decoding is memory-bound on GPUs, and how hybrid bonding shrinks the data path from millimeters to micrometers. That cuts access energy by 77 to 83 percent compared with HBM. It covers how the design differs from processing-in-memory, and the trade-offs of small capacity and a heat limit of 85 degrees Celsius. The discussion shows why existing tools like Ramulator and earlier accelerator simulators fall short. It also goes through DRAM details such as logical rows, row-buffer locality, channel interleaving, and the tension block-paged KV cache adds. The episode ends on a skeptical question: how far conclusions can be trusted beyond the specific chip points that were validated. Sources: 1. A Full-Stack Performance Evaluation Infrastructure for 3D-DRAM-based LLM Accelerators — Cong Li, Chenhao Xue, Yi Ren, Xiping Dong, Yu Cheng, Yinbo Hu, Fujun Bai, Yixin Guo, Xiping Jiang, Qiang Wu, Zhi Yang, Zhe Cheng, Yuan Xie, Guangyu Sun, 2026 http://arxiv.org/abs/2604.08044v1 2. Tetris: Scalable and Efficient Neural Network Acceleration with 3D Memory — Mingyu Gao, Jing Pu, Xuan Yang, Mark Horowitz, Christos Kozyrakis, 2017 https://scholar.google.com/scholar?q=Tetris%3A+Scalable+and+Efficient+Neural+Network+Acceleration+with+3D+Memory 3. Neurocube: A Programmable Digital Neuromorphic Architecture with High-Density 3D Memory — Duckhwan Kim, Jaeha Kung, Sek Chai, Sudhakar Yalamanchili, Saibal Mukhopadhyay, 2016 https://scholar.google.com/scholar?q=Neurocube%3A+A+Programmable+Digital+Neuromorphic+Architecture+with+High-Density+3D+Memory 4. AttAcc! Unleashing the Power of PIM for Batched Transformer-based Generative Model Inference — Jaehyun Park, Jaewan Choi, Kwanhee Kyung, Michael Jaemin Kim, Yongsuk Kwon, Nam Sung Kim, Jung Ho Ahn, 2024 https://scholar.google.com/scholar?q=AttAcc%21+Unleashing+the+Power+of+PIM+for+Batched+Transformer-based+Generative+Model+Inference 5. A 1ynm 1.25V 8Gb, 16Gb/s/pin GDDR6-based Accelerator-in-Memory supporting 1TFLOPS MAC Operation and Various Activation Functions for Deep-Learning Applications — Seongju Lee, Sukhan Lee, et al. (SK hynix), 2022 https://scholar.google.com/scholar?q=A+1ynm+1.25V+8Gb%2C+16Gb%2Fs%2Fpin+GDDR6-based+Accelerator-in-Memory+supporting+1TFLOPS+MAC+Operation+and+Various+Activation+Functions+for+Deep-Learning+Applications 6. The gem5 Simulator — Nathan Binkert, Bradford Beckmann, Gabriel Black, Steven Reinhardt, et al., 2011 https://scholar.google.com/scholar?q=The+gem5+Simulator 7. Ramulator: A Fast and Extensible DRAM Simulator — Yoongu Kim, Weikun Yang, Onur Mutlu, 2016 https://scholar.google.com/scholar?q=Ramulator%3A+A+Fast+and+Extensible+DRAM+Simulator 8. Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling — Mahmoud Khairy, Zhesheng Shen, Tor M. Aamodt, Timothy G. Rogers, 2020 https://scholar.google.com/scholar?q=Accel-Sim%3A+An+Extensible+Simulation+Framework+for+Validated+GPU+Modeling 9. LLMCompass: Enabling Efficient Hardware Design for Large Language Model Inference — Hengrui Zhang, August Ning, Rohan Baskar Prabhakar, David Wentzlaff, 2024 https://scholar.google.com/scholar?q=LLMCompass%3A+Enabling+Efficient+Hardware+Design+for+Large+Language+Model+Inference 10. Halide: A Language and Compiler for Optimizing Parallelism, Locality, and Recomputation in Image Processing Pipelines — Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, Saman Amarasinghe, 2013 https://scholar.google.com/scholar?q=Halide%3A+A+Language+and+Compiler+for+Optimizing+Parallelism%2C+Locality%2C+and+Recomputation+in+Image+Processing+Pipelines 11. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning — Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, Arvind Krishnamurthy, 2018 https://scholar.google.com/scholar?q=TVM%3A+An+Automated+End-to-End+Optimizing+Compiler+for+Deep+Learning 12. Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations — Philippe Tillet, H. T. Kung, David Cox, 2019 https://scholar.google.com/scholar?q=Triton%3A+An+Intermediate+Language+and+Compiler+for+Tiled+Neural+Network+Computations 13. Spatial: A Language and Compiler for Application Accelerators — David Koeplinger, Matthew Feldman, Raghu Prabhakar, Yaqi Zhang, Stefan Hadjis, Ruben Fiszel, Tian Zhao, Luigi Nardi, Ardavan Pedram, Christos Kozyrakis, Kunle Olukotun, 2018 https://scholar.google.com/scholar?q=Spatial%3A+A+Language+and+Compiler+for+Application+Accelerators 14. Stratum: System-Hardware Co-Design with Tiered Monolithic 3D-Stackable DRAM for Efficient MoE Serving — Yue Pan et al., 2025 https://scholar.google.com/scholar?q=Stratum%3A+System-Hardware+Co-Design+with+Tiered+Monolithic+3D-Stackable+DRAM+for+Efficient+MoE+Serving 15. H2-LLM: Hardware-Dataflow Co-Exploration for Heterogeneous Hybrid-Bonding-based Low-Batch LLM Inference — Cong Li, Yihan Yin, et al., 2025 https://scholar.google.com/scholar?q=H2-LLM%3A+Hardware-Dataflow+Co-Exploration+for+Heterogeneous+Hybrid-Bonding-based+Low-Batch+LLM+Inference 16. Ramulator 2.0: A Modern, Modular, and Extensible DRAM Simulator — Haocong Luo et al., 2023 https://scholar.google.com/scholar?q=Ramulator+2.0%3A+A+Modern%2C+Modular%2C+and+Extensible+DRAM+Simulator 17. Vidur: A Large-Scale Simulation Framework for LLM Inference — Amey Agrawal et al., 2024 https://scholar.google.com/scholar?q=Vidur%3A+A+Large-Scale+Simulation+Framework+for+LLM+Inference 18. Timeloop: A Systematic Approach to DNN Accelerator Evaluation — Angshuman Parashar et al., 2019 https://scholar.google.com/scholar?q=Timeloop%3A+A+Systematic+Approach+to+DNN+Accelerator+Evaluation 19. HotSpot 7.0 / Thermal Modeling for 3D ICs — Skadron et al.; Zhang et al., 2004-2020 https://scholar.google.com/scholar?q=HotSpot+7.0+%2F+Thermal+Modeling+for+3D+ICs 20. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon et al., 2023 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention 21. Samsung HBM-PIM / SK hynix AiM / AttAcc / NeuPIMs — Kwon et al.; Park et al.; Heo et al., 2021-2024 https://scholar.google.com/scholar?q=Samsung+HBM-PIM+%2F+SK+hynix+AiM+%2F+AttAcc+%2F+NeuPIMs 22. Splitwise / DistServe / Mooncake (PD disaggregation) — Patel et al.; Zhong et al.; Qin et al., 2024 https://scholar.google.com/scholar?q=Splitwise+%2F+DistServe+%2F+Mooncake+%28PD+disaggregation%29 23. DeepSeek-V3 / MLA technical reports — DeepSeek-AI, 2024-2025 https://scholar.google.com/scholar?q=DeepSeek-V3+%2F+MLA+technical+reports Interactive Visualization: ATLAS: Full-Stack Simulator for 3D-DRAM LLM Accelerators

  7. 2d ago

    AttnComp: Attention-Guided Adaptive Context Compression for RAG

    This episode examines AttnComp, a method that uses a language model's own middle-layer attention, lightly fine-tuned, to compress retrieved documents in retrieval-augmented generation. It covers why large retrieved contexts hurt: noisy passages can push accuracy below no retrieval, key facts get lost in the middle, and every extra token adds cost. It contrasts slow abstractive compressors with extractive ones that use fixed budgets and score each sentence alone, which misses bridge documents in multi-hop questions. The discussion explains how attention acts as a shared budget, so document scores compete jointly, and how a nucleus-sampling-style rule keeps as many documents as the attention requires, including none. The attention share on the instruction and first token serves as a signal for how relevant the context is. The episode also flags caveats, including attention sinks and the warning that attention is not always an explanation. Sources: 1. AttnComp: Attention-Guided Adaptive Context Compression for Retrieval-Augmented Generation — Lvzhou Luo, Yixuan Cao, Ping Luo, 2025 http://arxiv.org/abs/2509.17486 2. RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation — Fangyuan Xu, Weijia Shi, Eunsol Choi, 2023 https://scholar.google.com/scholar?q=RECOMP%3A+Improving+Retrieval-Augmented+LMs+with+Compression+and+Selective+Augmentation 3. LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models (and LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression) — Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu (Microsoft), 2023 https://scholar.google.com/scholar?q=LLMLingua%3A+Compressing+Prompts+for+Accelerated+Inference+of+Large+Language+Models+%28and+LongLLMLingua%3A+Accelerating+and+Enhancing+LLMs+in+Long+Context+Scenarios+via+Prompt+Compression%29 4. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, 2023 (TACL 2024) https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 5. Making Retrieval-Augmented Language Models Robust to Irrelevant Context — Ori Yoran, Tomer Wolfson, Ori Ram, Jonathan Berant, 2024 (ICLR) https://scholar.google.com/scholar?q=Making+Retrieval-Augmented+Language+Models+Robust+to+Irrelevant+Context 6. Distilling Knowledge from Reader to Retriever for Question Answering — Gautier Izacard, Edouard Grave, 2020 (ICLR 2021) https://scholar.google.com/scholar?q=Distilling+Knowledge+from+Reader+to+Retriever+for+Question+Answering 7. Retrieval Head Mechanistically Explains Long-Context Factuality — Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, Yao Fu, 2024 https://scholar.google.com/scholar?q=Retrieval+Head+Mechanistically+Explains+Long-Context+Factuality 8. Attention is not Explanation — Sarthak Jain, Byron C. Wallace, 2019 https://scholar.google.com/scholar?q=Attention+is+not+Explanation 9. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models — Zhenyu Zhang, Ying Sheng, Tianyi Zhou, et al., 2023 https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+Large+Language+Models 10. The Curious Case of Neural Text Degeneration — Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi, 2019 (ICLR 2020) https://scholar.google.com/scholar?q=The+Curious+Case+of+Neural+Text+Degeneration 11. Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity — Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, Jong C. Park, 2024 (NAACL) https://scholar.google.com/scholar?q=Adaptive-RAG%3A+Learning+to+Adapt+Retrieval-Augmented+Large+Language+Models+through+Question+Complexity 12. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection — Akari Asai, Zeqiu Wu, Ramakanth Pasunuru, Hannaneh Hajishirzi, et al., 2023 (ICLR 2024) https://scholar.google.com/scholar?q=Self-RAG%3A+Learning+to+Retrieve%2C+Generate%2C+and+Critique+through+Self-Reflection 13. Efficient Streaming Language Models with Attention Sinks — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2023 https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks 14. EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation — Taeho Hwang, Sukmin Cho, Soyeong Jeong, Hoyun Song, SeungYoon Han, Jong C. Park, 2024 https://scholar.google.com/scholar?q=EXIT%3A+Context-Aware+Extractive+Compression+for+Enhancing+Retrieval-Augmented+Generation 15. Provence: efficient and robust context pruning for retrieval-augmented generation — Nadezhda Chirkova, Thibault Formal, Vassilina Nikoulina, Stéphane Clinchant, 2025 https://scholar.google.com/scholar?q=Provence%3A+efficient+and+robust+context+pruning+for+retrieval-augmented+generation 16. SnapKV: LLM Knows What You Are Looking For Before Generation — Yuhong Li et al., 2024 https://scholar.google.com/scholar?q=SnapKV%3A+LLM+Knows+What+You+Are+Looking+For+Before+Generation 17. The Llama 3 Herd of Models / Mistral 7B / Qwen2 — Grattafiori et al.; Jiang et al.; Yang et al., 2023-2024 https://scholar.google.com/scholar?q=The+Llama+3+Herd+of+Models+%2F+Mistral+7B+%2F+Qwen2 Interactive Visualization: AttnComp: Attention-Guided Adaptive Context Compression for RAG

  8. 2d ago

    Context Language Models: Letting the Model Rewrite Its Own Context

    This episode explores "Context Language Models," a paper that lets a model rewrite its own context window with ordinary Bash or Python edits, instead of relying on harness-defined truncation or a fixed menu of context actions. It covers the three tiers of context control, the serving catch (an edit in the middle or at the start of the context can cost several times more prefill compute than an append, because it breaks the shared prefix), and the ContextBench diagnostic. On the BrowseComp-Plus deep-research benchmark, the approach scores 59.4 percent, about six points above a summary-compaction baseline, with roughly a fifth less compute. The discussion also covers the models' emergent behavior, including an orchestrator that updates a subagent scoreboard in place 163 times and a model that writes and calls its own compaction function. Listeners get a critical look at whether these gains hold up, given that the cost metric is analytic FLOPs rather than measured latency. Sources: 1. Context Language Models — Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert, Teng Xiao, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, Pang Wei Koh, 2026 http://arxiv.org/abs/2609.37725 2. MemGPT: Towards LLMs as Operating Systems — Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez, 2023 https://scholar.google.com/scholar?q=MemGPT%3A+Towards+LLMs+as+Operating+Systems 3. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, 2023 (TACL 2024) https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts 4. Recursive Language Models — Alex L. Zhang, Tim Kraska, Omar Khattab, 2025 https://scholar.google.com/scholar?q=Recursive+Language+Models 5. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models (ACE) — Qizheng Zhang et al. (Stanford, SambaNova and collaborators), 2025 https://scholar.google.com/scholar?q=Agentic+Context+Engineering%3A+Evolving+Contexts+for+Self-Improving+Language+Models+%28ACE%29 6. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, Ion Stoica, 2023 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention 7. SGLang: Efficient Execution of Structured Language Model Programs (RadixAttention) — Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, Ying Sheng, 2023 (NeurIPS 2024) https://scholar.google.com/scholar?q=SGLang%3A+Efficient+Execution+of+Structured+Language+Model+Programs+%28RadixAttention%29 8. CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion — Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, Junchen Jiang, 2024 (EuroSys 2025) https://scholar.google.com/scholar?q=CacheBlend%3A+Fast+Large+Language+Model+Serving+for+RAG+with+Cached+Knowledge+Fusion 9. Prompt Cache: Modular Attention Reuse for Low-Latency Inference — In Gim, Guojun Chen, Seung-seob Lee, Nikhil Sarda, Anurag Khandelwal, Lin Zhong, 2023 (MLSys 2024) https://scholar.google.com/scholar?q=Prompt+Cache%3A+Modular+Attention+Reuse+for+Low-Latency+Inference 10. EPIC: Efficient Position-Independent Caching for Serving Large Language Models — Junhao Hu, Wenrui Huang, Weidong Wang, et al., 2025 https://scholar.google.com/scholar?q=EPIC%3A+Efficient+Position-Independent+Caching+for+Serving+Large+Language+Models 11. Let the Code LLM Edit Itself When You Edit the Code (PIE) — Zhenyu He, Jun Zhang, Shengjie Luo, Jingjing Xu, Zhi Zhang, Di He, 2025 https://scholar.google.com/scholar?q=Let+the+Code+LLM+Edit+Itself+When+You+Edit+the+Code+%28PIE%29 12. Memento: Teaching LLMs to Manage Their Own Context — Vasilis Kontonis, Yuchen Zeng, Shivam Garg, et al., 2026 https://scholar.google.com/scholar?q=Memento%3A+Teaching+LLMs+to+Manage+Their+Own+Context 13. Scaling Long-Horizon LLM Agent via Context-Folding — Weiwei Sun, Miao Lu, Zhan Ling, et al., 2025 https://scholar.google.com/scholar?q=Scaling+Long-Horizon+LLM+Agent+via+Context-Folding 14. MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents — Zijian Zhou, Ao Qu, Zhaoxuan Wu, et al., 2026 https://scholar.google.com/scholar?q=MEM1%3A+Learning+to+Synergize+Memory+and+Reasoning+for+Efficient+Long-Horizon+Agents 15. Meta-Harness: End-to-End Optimization of Model Harnesses — Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, Chelsea Finn, 2026 https://scholar.google.com/scholar?q=Meta-Harness%3A+End-to-End+Optimization+of+Model+Harnesses 16. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning — Lakshya A. Agrawal, et al., 2026 https://scholar.google.com/scholar?q=GEPA%3A+Reflective+Prompt+Evolution+Can+Outperform+Reinforcement+Learning 17. Efficient Memory Management for Large Language Model Serving with PagedAttention; and SGLang / RadixAttention — Kwon et al. (vLLM); Lianmin Zheng et al. (SGLang), 2023/2024 https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention%3B+and+SGLang+%2F+RadixAttention 18. Self-Generated Prompt Injections in Compaction Summaries — OpenAI, 2026 https://scholar.google.com/scholar?q=Self-Generated+Prompt+Injections+in+Compaction+Summaries 19. Training Language Models to Reason Efficiently; and Think Dense, Not Long (DDCA) — Daman Arora, Andrea Zanette; Keqin Peng et al., 2026 https://scholar.google.com/scholar?q=Training+Language+Models+to+Reason+Efficiently%3B+and+Think+Dense%2C+Not+Long+%28DDCA%29 20. Titans / Mamba / Gated DeltaNet and learned-memory architectures — Behrouz et al.; Gu & Dao; Yang et al., 2024–2025 https://scholar.google.com/scholar?q=Titans+%2F+Mamba+%2F+Gated+DeltaNet+and+learned-memory+architectures Interactive Visualization: Context Language Models: Letting the Model Rewrite Its Own Context

See All (969)

Ratings & Reviews

3.7
out of 5
3 Ratings

About

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

Information

You Might Also Like