Abstracts: January 25, 2024 Microsoft Research Podcast

Members of the research community at Microsoft work continuously to advance their respective fields. Abstracts brings its audience to the cutting edge with them through short, compelling conversations about new and noteworthy achievements.In this episode, Senior Researchers Jordan Ash (https://www.microsoft.com/en-us/research/people/joash/) and Dipendra Misra (https://www.microsoft.com/en-us/research/people/dimisra/) join host Gretchen Huizinga to discuss “The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction (https://www.microsoft.com/en-us/research/publication/the-truth-is-in-there-improving-reasoning-in-language-models-with-layer-selective-rank-reduction/),” which was accepted to the 2024 International Conference on Learning Representations (ICLR). Layer-Selective Rank reduction, or LASER, is an intervention for targeted parameter reduction in transformer-based models. The work shows that the removal of certain parameters not only maintains model performance like some existing parameter-reduction methods but can actually improve it—no additional training necessary.To learn more about the paper and related topics, register for Microsoft Research Forum (https://aka.ms/researchforum/), a series of panel discussions and lightning talks around science and technology research in the era of general AI.Learn more:* The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction (https://www.microsoft.com/en-us/research/publication/the-truth-is-in-there-improving-reasoning-in-language-models-with-layer-selective-rank-reduction/) | Publication, December 2023* LASER code on GitHub (https://pratyushasharma.github.io/laser/)