AI Post Transformers

mcgrof

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

  1. 1 day ago

    Naive Test-Time Adaptation Destabilizes LLM Predictions

    This episode explores a paper proposing SCALENET, a hypernetwork-based fix for unsupervised test-time adaptation (TTA) in large language models. It examines why naive per-prompt gradient updates are unstable — a 70-billion-parameter Llama model's negative log-likelihood balloons from 2.21 to 11.49 after just five adaptation steps — and traces the problem to high-variance single-sample gradients that can't average out the way batch training does. The discussion covers the constrained "adapt-and-reset" setup used in real deployment, where models take a few unsupervised gradient steps on LoRA attention matrices per prompt before discarding the update, and explains why a single global learning rate can't work when small rates do nothing and large ones destroy the model. Listeners interested in the mechanics of on-the-fly model adaptation, LoRA-based efficient tuning, and the control-theory-like challenge of stabilizing per-layer, per-step learning rates will find the breakdown of the failure modes and the proposed hypernetwork solution especially compelling. Sources: 1. Unsupervised Layer-Wise Dynamic Test Time Adaptation for LLMs — Longhuan Xu, Cunjian Chen, Feng Yin, 2026 http://arxiv.org/abs/2602.09719 2. Test-Time Training with Self-Supervision for Generalization under Distribution Shifts — Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, Moritz Hardt, 2020 https://scholar.google.com/scholar?q=Test-Time+Training+with+Self-Supervision+for+Generalization+under+Distribution+Shifts 3. Tent: Fully Test-Time Adaptation by Entropy Minimization — Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, Trevor Darrell, 2021 https://scholar.google.com/scholar?q=Tent%3A+Fully+Test-Time+Adaptation+by+Entropy+Minimization 4. Test-Time Training on Nearest Neighbors for Large Language Models — Moritz Hardt, Yu Sun, 2024 https://scholar.google.com/scholar?q=Test-Time+Training+on+Nearest+Neighbors+for+Large+Language+Models 5. The Surprising Effectiveness of Test-Time Training for Abstract Reasoning — Ekin Akyürek, Mehul Damani, Linlu Qiu, Han Guo, Yoon Kim, Jacob Andreas, 2024 https://scholar.google.com/scholar?q=The+Surprising+Effectiveness+of+Test-Time+Training+for+Abstract+Reasoning 6. Test-time Learning for Large Language Models — Hu, J., Zhang, Z., Chen, G., Wen, X., Shuai, C., Luo, W., Xiao, B., Li, Y., Tan, M., 2025 https://scholar.google.com/scholar?q=Test-time+Learning+for+Large+Language+Models 7. SLOT: Sample-specific Language Model Optimization at Test-time — Hu, Y., Zhang, X., Fang, X., Chen, Z., Wang, X., Zhang, H., Qi, G., 2025 https://scholar.google.com/scholar?q=SLOT%3A+Sample-specific+Language+Model+Optimization+at+Test-time 8. COME: Test-time Adaption by Conservatively Minimizing Entropy — Zhang, Q., Bian, Y., Kong, X., Zhao, P., Zhang, C., 2024 https://scholar.google.com/scholar?q=COME%3A+Test-time+Adaption+by+Conservatively+Minimizing+Entropy 9. Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models — Rannen-Triki, A., Bornschein, J., Pascanu, R., Hutter, M., et al., 2024 https://scholar.google.com/scholar?q=Revisiting+Dynamic+Evaluation%3A+Online+Adaptation+for+Large+Language+Models 10. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML) — Finn, C., Abbeel, P., Levine, S., 2017 https://scholar.google.com/scholar?q=Model-Agnostic+Meta-Learning+for+Fast+Adaptation+of+Deep+Networks+%28MAML%29 Interactive Visualization: Naive Test-Time Adaptation Destabilizes LLM Predictions

  2. 2 days ago

    Continual Learning in LLMs: Beyond Catastrophic Forgetting

    This episode explores a survey on continual learning in large language models, examining how models can be updated after pretraining without the prohibitive cost of full retraining or the risk of catastrophic forgetting — the phenomenon where new training quietly degrades performance on tasks a model previously handled well. The discussion breaks down the problem across three distinct LLM training stages (pretraining, fine-tuning, and alignment) and maps them onto three classical mitigation strategies: rehearsal-based methods that replay old data, regularization-based methods that penalize changes to critical parameters, and architecture-based methods that add task-specific capacity like adapters or LoRA modules while freezing the rest. The hosts debate the survey's core organizational claim — that structuring the literature by mechanism rather than by application domain (medical, legal, financial) offers a more useful lens for practitioners trying to borrow a specific forgetting-mitigation technique. Listeners interested in the practical tradeoffs of keeping frontier models current — especially around data that can never legally enter a pretraining corpus, like medical or financial records — will find this a grounded framing of a problem every deployed LLM eventually faces. Sources: 1. Continual Learning in Large Language Models: Methods, Challenges, and Opportunities — Hongyang Chen, Zhongwu Sun, Hongfei Ye, Kunchi Li, Xuemin Lin, 2026 http://arxiv.org/abs/2603.12658 2. Don't Stop Pretraining: Adapt Language Models to Domains and Tasks — Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith, 2020 https://scholar.google.com/scholar?q=Don%27t+Stop+Pretraining%3A+Adapt+Language+Models+to+Domains+and+Tasks 3. Simple and Scalable Strategies to Continually Pre-train Large Language Models — Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats L. Richter, Quentin Anthony, Timothée Lesort, Eugene Belilovsky, Irina Rish, 2024 https://scholar.google.com/scholar?q=Simple+and+Scalable+Strategies+to+Continually+Pre-train+Large+Language+Models 4. LLaMA Pro: Progressive LLaMA with Block Expansion — Chengyue Wu, Yukang Gan, Yixiao Ge, Zeyu Lu, Jiahao Wang, Ye Feng, Ying Shan, Ping Luo, 2024 https://scholar.google.com/scholar?q=LLaMA+Pro%3A+Progressive+LLaMA+with+Block+Expansion 5. Code Llama: Open Foundation Models for Code — Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, and the Code Llama team at Meta AI, 2023 https://scholar.google.com/scholar?q=Code+Llama%3A+Open+Foundation+Models+for+Code 6. Editing Models with Task Arithmetic — Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, Ali Farhadi, 2023 https://scholar.google.com/scholar?q=Editing+Models+with+Task+Arithmetic 7. TIES-Merging: Resolving Interference When Merging Models — Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, Mohit Bansal, 2023 https://scholar.google.com/scholar?q=TIES-Merging%3A+Resolving+Interference+When+Merging+Models 8. Overcoming Catastrophic Forgetting in Neural Networks (EWC) — James Kirkpatrick et al., 2017 https://scholar.google.com/scholar?q=Overcoming+Catastrophic+Forgetting+in+Neural+Networks+%28EWC%29 Interactive Visualization: Continual Learning in LLMs: Beyond Catastrophic Forgetting

  3. 2 days ago

    Hal: Memory-Augmented LLM Agents Still Hit Continual Learning's Wall

    This episode explores a paper examining what happens to continual learning problems when LLM agents shift from parametric updates to memory-augmented architectures. Rather than accepting the industry assumption that external memory sidesteps catastrophic forgetting entirely, the researchers run classic continual-learning protocols on memory-based agents and find the same core problem resurfaces in a new form — shifting from parameter capacity to context-window retrieval capacity. They identify three specific failure modes: retrieval pollution (irrelevant memories crowding the prompt), context competition (useful memories getting displaced by other retrieved items), and memory dilution (relevant material becoming harder to surface as the memory store grows). The discussion traces this argument against the history of catastrophic forgetting and prior mitigation techniques like Elastic Weight Consolidation and Gradient Episodic Memory, then explains how the paper reframes the stability-plasticity dilemma for retrieval-based systems. Listeners interested in agent design, RAG architectures, or the assumptions underlying memory-augmented LLMs will find the paper's reframing — that memory doesn't eliminate the bottleneck, it just relocates it — a useful corrective to a widely repeated industry pitch. Sources: 1. Hal: Memory-Augmented LLM Agents Still Hit Continual Learning's Wall https://arxiv.org/pdf/2604.27003 2. A-Mem: Agentic Memory for LLM Agents — Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, Yongfeng Zhang, 2025 https://scholar.google.com/scholar?q=A-Mem%3A+Agentic+Memory+for+LLM+Agents 3. MemGPT: Towards LLMs as Operating Systems — Charles Packer, Vivian Fang, Shishir G. Patil, Kevin Lin, Sarah Wooders, Joseph E. Gonzalez, 2023 https://scholar.google.com/scholar?q=MemGPT%3A+Towards+LLMs+as+Operating+Systems 4. How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior — Zidi Xiong, Yuping Lin, Wenya Xie, Pengfei He, Zirui Liu, Jiliang Tang, Himabindu Lakkaraju, Zhen Xiang, 2025 https://scholar.google.com/scholar?q=How+Memory+Management+Impacts+LLM+Agents%3A+An+Empirical+Study+of+Experience-Following+Behavior 5. From RAG to Memory: Non-Parametric Continual Learning for Large Language Models — Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, Yu Su, 2025 https://scholar.google.com/scholar?q=From+RAG+to+Memory%3A+Non-Parametric+Continual+Learning+for+Large+Language+Models 6. The Probabilistic Relevance Framework: BM25 and Beyond — Stephen Robertson, Hugo Zaragoza, 2009 https://scholar.google.com/scholar?q=The+Probabilistic+Relevance+Framework%3A+BM25+and+Beyond Interactive Visualization: Hal: Memory-Augmented LLM Agents Still Hit Continual Learning's Wall

  4. 2 days ago

    In-Place Test-Time Training Turns Fast Weights Into Online Memory

    This episode explores a new test-time training method called In-Place TTT, which repurposes the down-projection matrix inside a model's existing gated MLP as adaptable "fast weights," letting a pretrained model keep learning during inference without any architectural changes. A key innovation is replacing the reconstruction-style training target used in prior TTT approaches with an LM-aligned target built from a causal convolution over token embeddings, which the authors prove (via an induction-head theorem) actually raises the probability of the correct next token. The discussion covers how a context-parallel scan preserves causality while enabling parallel computation of these updates, and walks through benchmark results showing the method trailing a baseline at short context but pulling substantially ahead as sequence length grows, tested across Qwen3-4B, LLaMA-3.1-8B, and Qwen3-14B. The hosts also dig into an ablation showing that mid-sized chunk sizes outperform larger ones — a counterintuitive result tied to how often the fast weights get to update rather than raw parallelism — plus efficiency data showing the approach barely affects throughput or memory. It's a concrete look at how far you can push adaptive inference-time learning while reusing a model's own existing structure. Sources: 1. In-Place Test-Time Training — Guhao Feng, Shengjie Luo, Kai Hua, Ge Zhang, Di He, Wenhao Huang, Tianle Cai, 2026 http://arxiv.org/abs/2604.06169 2. Learning to (Learn at Test Time): RNNs with Expressive Hidden States — Yu Sun, Xinhao Li, Karan Dalal, et al., 2024 https://scholar.google.com/scholar?q=Learning+to+%28Learn+at+Test+Time%29%3A+RNNs+with+Expressive+Hidden+States 3. Test-Time Training Done Right (LaCT) — Tianyuan Zhang, Sai Bi, Yicong Hong, et al., 2025 https://scholar.google.com/scholar?q=Test-Time+Training+Done+Right+%28LaCT%29 4. Titans: Learning to Memorize at Test Time — Ali Behrouz, Peilin Zhong, Vahab Mirrokni, 2024 https://scholar.google.com/scholar?q=Titans%3A+Learning+to+Memorize+at+Test+Time 5. Transformer Feed-Forward Layers Are Key-Value Memories — Mor Geva, Roei Schuster, Jonathan Berant, Omer Levy, 2020 https://scholar.google.com/scholar?q=Transformer+Feed-Forward+Layers+Are+Key-Value+Memories 6. Locating and Editing Factual Associations in GPT (ROME) — Kevin Meng, David Bau, Alex Andonian, Yonatan Belinkov, 2022 https://scholar.google.com/scholar?q=Locating+and+Editing+Factual+Associations+in+GPT+%28ROME%29 7. LoRA: Low-Rank Adaptation of Large Language Models — Edward Hu, Yelong Shen, Phillip Wallis, et al., 2022 https://scholar.google.com/scholar?q=LoRA%3A+Low-Rank+Adaptation+of+Large+Language+Models Interactive Visualization: In-Place Test-Time Training Turns Fast Weights Into Online Memory

  5. 2 days ago

    Kohonen's 1972 Correlation Matrix Memory, Decades Before Attention

    This episode revisits Teuvo Kohonen's 1972 paper "Correlation Matrix Memories," which reframes associative memory as a hardware fault-tolerance problem rather than a representation-learning one. Kohonen builds a memory from outer-product sums of key and data vectors, then shows mathematically how much recall quality degrades when connections are randomly dropped (an "incomplete" correlation matrix memory) rather than fully wired. The discussion traces the paper's lineage against optical holography models and Steinbuch's Lernmatrix, and unpacks concepts like crosstalk and graceful degradation as information gets smeared additively across the matrix instead of stored in one fragile spot. A tangent draws — and partly disputes — a comparison between Kohonen's outer-product accumulation and the mechanics underlying modern attention, debating whether the resemblance is structural or purely coincidental given the total absence of learning or gradients in the original scheme. Listeners interested in the deep history of neural memory models and how old hardware constraints shaped ideas that echo in today's architectures will find plenty to chew on. Sources: 1. Kohonen's 1972 Correlation Matrix Memory, Decades Before Attention https://lucidar.me/fr/neural-networks/files/1972-correlation-matrix-memories.pdf 2. Neural networks and physical systems with emergent collective computational abilities — John J. Hopfield, 1982 https://scholar.google.com/scholar?q=Neural+networks+and+physical+systems+with+emergent+collective+computational+abilities 3. Non-Holographic Associative Memory — David Willshaw, O. P. Buneman, H. Christopher Longuet-Higgins, 1969 https://scholar.google.com/scholar?q=Non-Holographic+Associative+Memory 4. Hopfield Networks is All You Need — Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, et al., 2020 https://scholar.google.com/scholar?q=Hopfield+Networks+is+All+You+Need 5. Linear Transformers Are Secretly Fast Weight Programmers — Imanol Schlag, Kazuki Irie, Jürgen Schmidhuber, 2021 https://scholar.google.com/scholar?q=Linear+Transformers+Are+Secretly+Fast+Weight+Programmers 6. A Simple Neural Network Generating an Interactive Memory — James A. Anderson, 1972 https://scholar.google.com/scholar?q=A+Simple+Neural+Network+Generating+an+Interactive+Memory 7. Representation of Associated Data by Matrix Operators — Teuvo Kohonen, Matti Ruohonen, 1973 https://scholar.google.com/scholar?q=Representation+of+Associated+Data+by+Matrix+Operators 8. Sparse Distributed Memory — Pentti Kanerva, 1988 https://scholar.google.com/scholar?q=Sparse+Distributed+Memory 9. Die Lernmatrix — K. Steinbuch, 1961 https://scholar.google.com/scholar?q=Die+Lernmatrix 10. Associative holographic memories — D. Gabor, 1969 https://scholar.google.com/scholar?q=Associative+holographic+memories 11. A class of randomly organized associative memories — T. Kohonen, 1971 https://scholar.google.com/scholar?q=A+class+of+randomly+organized+associative+memories Interactive Visualization: Kohonen's 1972 Correlation Matrix Memory, Decades Before Attention

  6. 2 days ago

    Learning, Fast and Slow: LLMs That Adapt Without Forgetting

    This episode explores catastrophic forgetting and plasticity loss in RL-trained language models, and introduces "Fast-Slow Training," a method combining slow weight updates (RLVR) with fast in-context learning to address both. The hosts unpack the distinction between RLVR's automatic, verifiable rewards and traditional RLHF, then dig into two separate failure modes of pure RL post-training: models forgetting general competence while chasing a narrow reward signal, and a subtler loss of plasticity where updates leave models increasingly unable to absorb new tasks. Framing the two training channels as a System 1/System 2 split, the discussion centers on the paper's headline result — combining both channels reaches RL's peak accuracy with up to three times fewer samples, drifts up to seventy percent less from the base model, and preserves the capacity to learn subsequent tasks where pure RL stalls. Listeners interested in the mechanics and tradeoffs of continual learning in large language models will find a grounded walkthrough of why prompting alone hits a ceiling and why weight updates alone come with hidden costs. Sources: 1. Learning, Fast and Slow: Towards LLMs That Adapt Continually — Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal, Joseph E. Gonzalez, Matei Zaharia, Kurt Keutzer, Inderjit S Dhillon, Rishabh Agarwal, Devvrit Khatri, 2026 http://arxiv.org/abs/2605.12484 2. Loss of Plasticity in Deep Continual Learning — Shibhansh Dohare, J. Fernando Hernandez-Garcia, Qingfeng Lan, A. Rupam Mahmood, Richard S. Sutton, et al., 2024 (Nature; preprint circulated as 'Maintaining Plasticity via Continual Backprop' from 2021) https://scholar.google.com/scholar?q=Loss+of+Plasticity+in+Deep+Continual+Learning 3. On Warm-Starting Neural Network Training — Jordan T. Ash, Ryan P. Adams, 2020 (NeurIPS) https://scholar.google.com/scholar?q=On+Warm-Starting+Neural+Network+Training 4. The Primacy Bias in Deep Reinforcement Learning — Evgenii Nikishin, Max Schwarzer, Pierluca D'Oro, Pierre-Luc Bacon, Aaron Courville, 2022 (ICML) https://scholar.google.com/scholar?q=The+Primacy+Bias+in+Deep+Reinforcement+Learning 5. Understanding Plasticity in Neural Networks — Clare Lyle, Zeyu Zheng, Evgenii Nikishin, Bernardo Avila Pires, Razvan Pascanu, Will Dabney, 2023 (ICML) https://scholar.google.com/scholar?q=Understanding+Plasticity+in+Neural+Networks 6. RL's razor: Why online reinforcement learning forgets less — Idan Shenfeld, Jyothish Pari, Pulkit Agrawal, 2025 https://scholar.google.com/scholar?q=RL%27s+razor%3A+Why+online+reinforcement+learning+forgets+less 7. The Art of Scaling Reinforcement Learning Compute for LLMs (ScaleRL) — Devvrit Khatri, Lovish Madaan, Rishabh Tiwari, Rachit Bansal, Sai Surya Duvvuri, Manzil Zaheer, Inderjit S. Dhillon, David Brandfonbrener, Rishabh Agarwal, 2025 https://scholar.google.com/scholar?q=The+Art+of+Scaling+Reinforcement+Learning+Compute+for+LLMs+%28ScaleRL%29 8. Fine-tuning and prompt optimization: Two great steps that work better together (BetterTogether) — Dilara Soylu, Christopher Potts, Omar Khattab, 2024 https://scholar.google.com/scholar?q=Fine-tuning+and+prompt+optimization%3A+Two+great+steps+that+work+better+together+%28BetterTogether%29 9. Mitigating plasticity loss in continual reinforcement learning by reducing churn — Hongyao Tang, Johan Obando-Ceron, Pablo Samuel Castro, Aaron Courville, Glen Berseth, 2025 https://scholar.google.com/scholar?q=Mitigating+plasticity+loss+in+continual+reinforcement+learning+by+reducing+churn 10. What can you do when you have zero rewards during RL? — Jatin Prakash, Anirudh Buvanesh, 2025 https://scholar.google.com/scholar?q=What+can+you+do+when+you+have+zero+rewards+during+RL%3F Interactive Visualization: Learning, Fast and Slow: LLMs That Adapt Without Forgetting

  7. 2 days ago

    Model-Agnostic Meta-Learning for Fast Task Adaptation

    This episode explores Model-Agnostic Meta-Learning (MAML), the 2017 approach from Chelsea Finn, Pieter Abbeel, and Sergey Levine that trains a single, architecture-agnostic initialization capable of fast adaptation across image classification, regression, and reinforcement learning. Rather than learning a task-specific update rule like earlier recurrent meta-learners, MAML optimizes the starting weights themselves so that a few steps of ordinary gradient descent adapt them well to a brand-new task from minimal data, tested through Omniglot and MiniImagenet few-shot classification, sinusoid regression, and MuJoCo/2D navigation RL. The discussion breaks down the inner-loop/outer-loop structure, the second-order gradient-through-gradient math (Hessian-vector products) needed to backpropagate through the adaptation step, and how finite-difference approximations sidestep the third-derivative problem when TRPO is used as the RL meta-optimizer. Listeners get a clear walkthrough of N-way K-shot learning and why one image per class is such an extreme test of generalization, plus a grounded comparison to the more familiar pretrain-then-fine-tune workflow. It's a good listen for anyone curious how a deceptively simple idea — learn to be easy to fine-tune — unified meta-learning across problem types that previously required separate specialized systems. Sources: 1. Model-Agnostic Meta-Learning for Fast Task Adaptation https://proceedings.mlr.press/v70/finn17a/finn17a.pdf 2. Optimization as a Model for Few-Shot Learning — Sachin Ravi, Hugo Larochelle, 2017 https://scholar.google.com/scholar?q=Optimization+as+a+Model+for+Few-Shot+Learning 3. Meta-Learning with Memory-Augmented Neural Networks — Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, Timothy Lillicrap, 2016 https://scholar.google.com/scholar?q=Meta-Learning+with+Memory-Augmented+Neural+Networks 4. Matching Networks for One Shot Learning — Oriol Vinyals, Charles Blundell, Tim Lillicrap, Daan Wierstra, 2016 https://scholar.google.com/scholar?q=Matching+Networks+for+One+Shot+Learning 5. Learning to Learn by Gradient Descent by Gradient Descent — Marcin Andrychowicz et al., 2016 https://scholar.google.com/scholar?q=Learning+to+Learn+by+Gradient+Descent+by+Gradient+Descent 6. Trust Region Policy Optimization — John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, Philipp Moritz, 2015 https://scholar.google.com/scholar?q=Trust+Region+Policy+Optimization 7. RL2: Fast Reinforcement Learning via Slow Reinforcement Learning — Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel, 2016 https://scholar.google.com/scholar?q=RL2%3A+Fast+Reinforcement+Learning+via+Slow+Reinforcement+Learning Interactive Visualization: Model-Agnostic Meta-Learning for Fast Task Adaptation

  8. 2 days ago

    TwinQuant: Manifold-Constrained Low-Rank Decomposition for 4-Bit Quantization

    This episode explores TwinQuant, a 4-bit post-training quantization method for large language models that challenges a core assumption behind prior techniques like SVDQuant: that a weight matrix's important information can be captured in a small, fixed set of directions. The hosts explain how LLM weight outliers turn out to be spread across hundreds of directions rather than concentrated, forcing earlier low-rank decomposition approaches into an unwinnable tradeoff between speed and accuracy. They unpack TwinQuant's solution — learning the low-rank split itself via manifold optimization, using a true orthogonal (Stiefel manifold) rotation that folds cleanly into RMSNorm layers alongside a more flexible invertible (general linear) transform for layer-specific residual handling — plus a fused kernel designed to keep the approach fast at inference. Along the way, the conversation walks through foundational quantization vocabulary (PTQ, WxAy notation, mixed-precision splits) for listeners newer to the topic. It's a compelling listen for anyone tracking how far LLMs can be compressed without sacrificing accuracy, and why the math behind "which parts of a weight matrix matter" is more complicated than earlier compression work assumed. Sources: 1. TwinQuant: Learnable Subspace Decomposition for 4-Bit LLM Quantization — Haodong Wang, Junjie Liu, Zicong Hong, Qianli Liu, Jian Lin, Song Guo, Xu Chen, 2026 http://arxiv.org/abs/2606.01556 2. Optimization Algorithms on Matrix Manifolds — P.-A. Absil, R. Mahony, R. Sepulchre, 2008 https://scholar.google.com/scholar?q=Optimization+Algorithms+on+Matrix+Manifolds 3. SpinQuant: LLM Quantization with Learned Rotations — Zechun Liu, Changsheng Zhao, Igor Fedorov, et al. (Meta AI), 2024 https://scholar.google.com/scholar?q=SpinQuant%3A+LLM+Quantization+with+Learned+Rotations 4. QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs — Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, et al., 2024 https://scholar.google.com/scholar?q=QuaRot%3A+Outlier-Free+4-Bit+Inference+in+Rotated+LLMs 5. Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform — Jun Li, Fuxin Li, Sinisa Todorovic, 2020 https://scholar.google.com/scholar?q=Efficient+Riemannian+Optimization+on+the+Stiefel+Manifold+via+the+Cayley+Transform 6. SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models — Li, M., Lin, Y., Zhang, Z., Cai, T., Li, X., Guo, J., Xie, E., Meng, C., Zhu, J.-Y., Han, S., 2025 https://scholar.google.com/scholar?q=SVDQuant%3A+Absorbing+Outliers+by+Low-Rank+Components+for+4-Bit+Diffusion+Models 7. FlatQuant: Flatness Matters for LLM Quantization — Sun, Y., Liu, R., Bai, H., Bao, H., Zhao, K., Li, Y., Yu, X., Hou, L., Yuan, C., Jiang, X., et al., 2025 https://scholar.google.com/scholar?q=FlatQuant%3A+Flatness+Matters+for+LLM+Quantization 8. OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models — Shao, W., Chen, M., Zhang, Z., Xu, P., Zhao, L., Li, Z., Zhang, K., Gao, P., Qiao, Y., Luo, P., 2024 https://scholar.google.com/scholar?q=OmniQuant%3A+Omnidirectionally+Calibrated+Quantization+for+Large+Language+Models Interactive Visualization: TwinQuant: Manifold-Constrained Low-Rank Decomposition for 4-Bit Quantization

See All (842)

About

AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis, proper citations, and nerdy humor.

Information

You Might Also Like