132 episodes

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. The hosts are the members of the AllenNLP team at Allen Institute for AI. All views expressed belong to the hosts and guests and do not represent their employers.

NLP Highlights Allen Institute for Artificial Intelligence

    • Science
    • 4.6 • 19 Ratings

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. The hosts are the members of the AllenNLP team at Allen Institute for AI. All views expressed belong to the hosts and guests and do not represent their employers.

    131 - Opportunities and Barriers between HCI and NLP, with Nanna Inie and Leon Derczynski

    131 - Opportunities and Barriers between HCI and NLP, with Nanna Inie and Leon Derczynski

    What can NLP researchers learn from Human Computer Interaction (HCI) research? We chatted with Nanna Inie and Leon Derczynski to find out. We discussed HCI's research processes including methods of inquiry, the data annotation processes used in HCI, and how they are different from NLP, and the cognitive methods used in HCI for qualitative error analyses. We also briefly talked about the opportunities the field of HCI presents for NLP researchers.

    This discussion is based on the following paper: https://aclanthology.org/2021.hcinlp-1.16/

    Nanna Inie is a postdoctoral researcher and Leon Derczynski is an associate professor in CS at the IT University of Copenhagen.

    The hosts for this episode are Ana Marasović and Pradeep Dasigi.

    • 46 min
    130 - Linking human cognitive patterns to NLP Models, with Lisa Beinborn

    130 - Linking human cognitive patterns to NLP Models, with Lisa Beinborn

    In this episode, we talk with Lisa Beinborn, an assistant professor at Vrije Universiteit Amsterdam, about how to use human cognitive signals to improve and analyze NLP models. We start by discussing different kinds of cognitive signals—eye-tracking, EEG, MEG, and fMRI—and challenges associated with using them. We then turn to Lisa’s recent work connecting interpretability measures with eye-tracking data, which reflect the relative importance measures of different tokens in human reading comprehension. We discuss empirical results suggesting that eye-tracking signals correlate strongly with gradient-based saliency measures, but not attention, in NLP methods. We conclude with discussion of the implications of these findings, as well as avenues for future work.

    Papers discussed in this episode:
    Towards best practices for leveraging human language processing signals for natural language processing: https://api.semanticscholar.org/CorpusID:219309655
    Relative Importance in Sentence Processing: https://api.semanticscholar.org/CorpusID:235358922

    Lisa Beinborn’s webpage: https://beinborn.eu/

    The hosts for this episode are Alexis Ross and Pradeep Dasigi.

    • 44 min
    129 - Transformers and Hierarchical Structure, with Shunyu Yao

    129 - Transformers and Hierarchical Structure, with Shunyu Yao

    In this episode, we talk to Shunyu Yao about recent insights into how transformers can represent hierarchical structure in language. Bounded-depth hierarchical structure is thought to be a key feature of natural languages, motivating Shunyu and his coauthors to show that transformers can efficiently represent bounded-depth Dyck languages, which can be thought of as a formal model of the structure of natural languages. We went on to discuss some of the intuitive ideas that emerge from the proofs, connections to RNNs, and insights about positional encodings that may have practical implications. More broadly, we also touched on the role of formal languages and other theoretical tools in modern NLP.

    Papers discussed in this episode:

    - Self-Attention Networks Can Process Bounded Hierarchical Languages (https://arxiv.org/abs/2105.11115)
    - Theoretical Limitations of Self-Attention in Neural Sequence Models (https://arxiv.org/abs/1906.06755)
    - RNNs can generate bounded hierarchical languages with optimal memory (https://arxiv.org/abs/2010.07515)
    - On the Practical Computational Power of Finite Precision RNNs for Language Recognition (https://arxiv.org/abs/1805.04908)

    Shunyu Yao's webpage: https://ysymyth.github.io/

    The hosts for this episode are William Merrill and Matt Gardner.

    • 35 min
    128 - Dynamic Benchmarking, with Douwe Kiela

    128 - Dynamic Benchmarking, with Douwe Kiela

    We discussed adversarial dataset construction and dynamic benchmarking in this episode with Douwe Kiela, a research scientist at Facebook AI Research who has been working on a dynamic benchmarking platform called Dynabench. Dynamic benchmarking tries to address the issue of many recent datasets getting solved with little progress being made towards solving the corresponding tasks. The idea is to involve models in the data collection loop to encourage humans to provide data points that are hard for those models, thereby continuously collecting harder datasets. We discussed the details of this approach, and some potential caveats. We also discussed dynamic leaderboards, a recent addition to Dynabench that rank systems based on their utility given specific use cases.

    Papers discussed in this episode:
    1. Dynabench: Rethinking Benchmarking in NLP (https://www.semanticscholar.org/paper/Dynabench%3A-Rethinking-Benchmarking-in-NLP-Kiela-Bartolo/77a096d80eb4dd4ccd103d1660c5a5498f7d026b)
    2. Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking (https://www.semanticscholar.org/paper/Dynaboard%3A-An-Evaluation-As-A-Service-Platform-for-Ma-Ethayarajh/d25bb256e5b69f769a429750217b0d9ec1cf4d86)
    3. Adversarial NLI: A New Benchmark for Natural Language Understanding (https://www.semanticscholar.org/paper/Adversarial-NLI%3A-A-New-Benchmark-for-Natural-Nie-Williams/9d87300892911275520a4f7a5e5abf4f1c002fec)
    4. DynaSent: A Dynamic Benchmark for Sentiment Analysis (https://www.semanticscholar.org/paper/DynaSent%3A-A-Dynamic-Benchmark-for-Sentiment-Potts-Wu/284dfcf7f25ca87b2db235c6cdc848b4143d3923)

    Douwe Kiela's webpage: https://douwekiela.github.io/

    The hosts for this episode are Pradeep Dasigi and Alexis Ross.

    • 47 min
    127 - Masakhane and Participatory Research for African Languages, with Tosin Adewumi and Perez Ogayo

    127 - Masakhane and Participatory Research for African Languages, with Tosin Adewumi and Perez Ogayo

    We invited members of Masakhane, Tosin Adewumi and Perez Ogayo, to talk about their EMNLP Findings paper that discusses why typical research is limited for low-resourced NLP and how participatory research can help.  

    As a result of participatory research, Masakhane has many, many success stories: first datasets and benchmarks in African languages, first research on human evaluation specifically for MT for low-resource languages, etc. In this episode, we talked about one of them—MasakhaNER—in more detail.

    The hosts for this episode are Pradeep Dasigi and Ana Marasović.

    --------------------------

    Tosin Adewumi is a PhD student at the Luleå University of Technology in Sweden. His Twitter handle: @tosintwit

    Perez Ogayo is an undergrad student at the African Leadership University in Rwanda. Her Twitter handle: @a_ogayo

    Masakhane is a grassroots organization whose mission is to strengthen and spur NLP research in African languages, for Africans, by Africans: https://www.masakhane.io/

    Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages (Findings of EMNLP 2020): https://arxiv.org/abs/2010.02353

    MasakhaNER: Named Entity Recognition for African languages (AfricaNLP Workshop @ EACL 2021): https://arxiv.org/abs/2103.11811

    • 47 min
    126 - Optimizing Continuous Prompts for Generation, with Lisa Li

    126 - Optimizing Continuous Prompts for Generation, with Lisa Li

    We invited Lisa Li to talk about her recent work, Prefix-Tuning: Optimizing Continuous Prompts for Generation. Prefix tuning is a lightweight alternative to finetuning, and the idea is to tune only a fixed-length task-specific continuous vector, and to keep the pretrained transformer parameters frozen. We discussed how prefix tuning compares with finetuning and other efficient alternatives on two tasks in various experimental settings, and in what scenarios prefix tuning is preferable.

    Lisa is a Phd student at Stanford University. Lisa's webpage: https://xiangli1999.github.io/

    The hosts for this episode are Pradeep Dasigi and Ana Marasović.

    • 47 min

Customer Reviews

4.6 out of 5
19 Ratings

19 Ratings

SambersCurtis ,

The only NLP Podcast

It’s nice that they are covering NLP. I’m not able to find anyone else that is doing that specifically. However, the hosts always try to punch holes in the guests theories. So they are always on the defensive. It makes for really unpleasant discussions.

Top Podcasts In Science

Listeners Also Subscribed To