AI: post transformers

mcgrof

The transformer architecture revolutionized the world of Neural Networks. It was a springboard for what we know today as modern artificial intelligence. This podcast focuses on modern state of the art research paper reviews starting from the transformer and on.

  1. Dario Amodei: The Adolescence of Technology

    56분 전

    Dario Amodei: The Adolescence of Technology

    Dario Amodei views the rise of powerful AI as a "technological adolescence" for humanity. He outlines critical risks: autonomous misalignment, biological misuse, totalitarian control, and economic disruption. He advocates for Constitutional AI and global regulation to prevail. Source: January 2026 The Adolescence of Technology Anthropic Dario Amodei https://www.darioamodei.com/essay/the-adolescence-of-technology

    17분
  2. Dario Amodei: Machines of Loving Grace

    2시간 전

    Dario Amodei: Machines of Loving Grace

    Dario Amodei argues that powerful AI could catalyze a "compressed 21st century," achieving 100 years of progress in a decade. He envisions radical breakthroughs in biology, neuroscience, and economic development while emphasizing the need to defend liberal democracy. Source: October 2024 Machines of Loving Grace Anthropic Dario Amodei https://www.darioamodei.com/essay/machines-of-loving-grace

    17분
  3. LongCat: Scaling Embeddings Outperforms Scaling Experts in Language Models

    2시간 전

    LongCat: Scaling Embeddings Outperforms Scaling Experts in Language Models

    Researchers from the LongCat introduced LongCat-Flash-Lite on January 2026, demonstrating that scaling embeddings via N-gram layers outperforms increasing Mixture-of-Experts parameters in high-sparsity regimes. This architecture uses system optimizations and speculative decoding to boost inference speed. Source: January 2026 Scaling Embeddings Outperforms Scaling Experts in Language Models Meituan LongCat Team Hong Liu, Jiaqi Zhang, Chao Wang, Xing Hu, Linkun Lyu, Jiaqi Sun, Xurui Yang, Bo Wang, Fengcun Li, Yulei Qian, Lingtong Si, Yerui Sun, Rumei Li, Peng Pei, Yuchen Xie, Xunliang Cai https://arxiv.org/pdf/2601.21204

    17분
  4. ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference

    3시간 전

    ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference

    ChunkKV improves LLM efficiency by compressing the KV cache using semantic chunks rather than isolated tokens, preserving linguistic integrity. It features layer-wise index reuse to boost throughput by 26.5%. Separately, Expected Attention estimates future token importance. Source: February 2025 ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference The Hong Kong University of Science and Technology (Guangzhou), The Hong Kong University of Science and Technology, HKUST Fok Ying Tung Research Institute, Terminus Technologies Xiang Liu, Zhenheng Tang, Peijie Dong, Zeyu Li, Yue Liu, Bo Li, Xuming Hu, Xiaowen Chu https://arxiv.org/pdf/2502.00299.pdf

    19분
  5. DR. KERNEL: Reinforcement Learning for Optimized Triton Kernel Generation

    3시간 전

    DR. KERNEL: Reinforcement Learning for Optimized Triton Kernel Generation

    Researchers introduced DR. KERNEL, a 14B model for Triton kernel generation trained via reinforcement learning. To prevent reward hacking and lazy optimization, they developed KERNELGYM for execution-based feedback and TRLOO for unbiased multi-turn RL. It rivals GPT-5. Source: February 2026 DR. KERNEL: Reinforcement Learning Done Right for Triton Kernel Generations HKUST, TikTok, CUHK(SZ), NTU Wei Liu, Jiawei Xu, Yingru Li, Longtao Zheng, Tianjian Li, Qian Liu, Junxian He https://arxiv.org/pdf/2602.05885

    12분
  6. A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications

    3시간 전

    A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications

    Mixture of Experts (MoE) is a scalable architecture that uses a gating function to activate specialized expert networks dynamically. This "divide and conquer" approach enhances efficiency and interpretability in CV, NLP, and Reinforcement Learning paradigms. Source: March 2025 A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications University of Houston Siyuan Mu, Sen Lin https://arxiv.org/pdf/2503.07137

    17분
  7. Sapient Intelligence: Hierarchical Reasoning Model

    3시간 전

    Sapient Intelligence: Hierarchical Reasoning Model

    The Hierarchical Reasoning Model (HRM) is a recurrent architecture using high-level planning and low-level execution modules to achieve deep latent reasoning. It employs Q-learning for adaptive computation, training a halting mechanism to optimize thinking time. Source: June 2025 Hierarchical Reasoning Model Sapient Intelligence, Singapore; Tsinghua University Guan Wang, Jin Li, Yuhao Sun, Xing Chen, Changling Liu, Yue Wu, Meng Lu, Sen Song, Yasin Abbasi Yadkori https://arxiv.org/pdf/2506.21734

    15분
  8. Advances in Attention Distillation for Efficient Transformer Models

    3시간 전

    Advances in Attention Distillation for Efficient Transformer Models

    Recent research advances attention distillation to optimize transformers. HAD binarizes keys/queries for efficiency, while SHD aligns varying head counts. CompoDistill improves multimodal reasoning via visual alignment, and new losses transfer visual characteristics in diffusion models. Sources: 1) February 3 2025 Hamming Attention Distillation: Binarizing Keys and Queries for Efficient Long-Context Transformers Mark Horton, Tergel Molom-Ochir, Peter Liu, Bhavna Gopal, Chiyue Wei, Cong Guo, Brady Taylor, Deliang Fan, Shan X. Wang, Hai Li, Yiran Chen https://doi.org/10.48550/arXiv.2502.01770 2) February 11 2025 Optimizing Knowledge Distillation in Transformers: Enabling Multi-Head Attention without Alignment Barriers Zhaodong Bing, Linze Li, Jiajun Liang https://doi.org/10.48550/arXiv.2502.07436 3) February 27 2025 Attention Distillation: A Unified Approach to Visual Characteristics Transfer Yang Zhou, Xu Gao, Zichong Chen, Hui Huang https://doi.org/10.48550/arXiv.2502.20235 4) October 14 2025 CompoDistill: Attention Distillation for Compositional Reasoning in Multimodal LLMs Jiwan Kim, Kibum Kim, Sangwoo Seo, Chanyoung Park https://doi.org/10.48550/arXiv.2510.12184

    17분
모두 보기(390개)

소개

The transformer architecture revolutionized the world of Neural Networks. It was a springboard for what we know today as modern artificial intelligence. This podcast focuses on modern state of the art research paper reviews starting from the transformer and on.

정보

  • 제작진
    mcgrof
  • 방송 연도
    2025년 - 2026년
  • 에피소드
    390
  • 등급
    전체 연령 사용가
  • 저작권
    © mcgrof
  • 웹사이트 보기
    AI: post transformers