107 episódios

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. The hosts are Matt Gardner, Pradeep Dasigi (research scientists at the Allen Institute for Artificial Intelligence) and Waleed Ammar (research scientist at Google). All views expressed belong to the hosts and guests and do not represent their employers.

NLP Highlights Allen Institute for Artificial Intelligence

    • Ciência

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. The hosts are Matt Gardner, Pradeep Dasigi (research scientists at the Allen Institute for Artificial Intelligence) and Waleed Ammar (research scientist at Google). All views expressed belong to the hosts and guests and do not represent their employers.

    106 - Ethical Considerations In NLP Research, Emily Bender

    106 - Ethical Considerations In NLP Research, Emily Bender

    In this episode, we talked to Emily Bender about the ethical considerations in developing NLP models and putting them in production. Emily cited specific examples of ethical issues, and talked about the kinds of potential concerns to keep in mind, both when releasing NLP models that will be used by real people, and also while conducting NLP research. We concluded by discussing a set of open-ended questions about designing tasks, collecting data, and publishing results, that Emily has put together towards addressing these concerns.

    Emily M. Bender is a Professor in the Department of Linguistics and an Adjunct Professor in the Department of Computer Science and Engineering at the University of Washington. She’s active on Twitter at @emilymbender.

    • 39 min
    105 - Question Generation, with Sudha Rao

    105 - Question Generation, with Sudha Rao

    In this episode we invite Sudha Rao to talk about question generation. We talk about different settings where you might want to generate questions: for human testing scenarios (rare), for data augmentation (has been done a bunch for SQuAD-like tasks), for detecting missing information / asking clarification questions, for dialog uses, and others. After giving an overview of the general area, we talk about the specifics of some of Sudha's work, including her ACL 2018 best paper on ranking clarification questions using EVPI. We conclude with a discussion of evaluating question generation, which is a hard problem, and what the exciting open questions there are in this research area.

    Sudha's website: https://raosudha.weebly.com/

    • 42 min
    104 - Model Distillation, with Victor Sanh and Thomas Wolf

    104 - Model Distillation, with Victor Sanh and Thomas Wolf

    In this episode we talked with Victor Sanh and Thomas Wolf from HuggingFace about model distillation, and DistilBERT as one example of distillation. The idea behind model distillation is compressing a large model by building a smaller model, with much fewer parameters, that approximates the output distribution of the original model, typically for increased efficiency. We discussed how model distillation was typically done previously, and then focused on the specifics of DistilBERT, including training objective, empirical results, ablations etc. We finally discussed what kinds of information you might lose when doing model distillation.

    • 31 min
    103 - Processing Language in Social Media, with Brendan O'Connor

    103 - Processing Language in Social Media, with Brendan O'Connor

    We talked to Brendan O’Connor for this episode about processing language in social media. Brendan started off by telling us about his projects that studied the linguistic and geographical patterns of African American English (AAE), and how obtaining data from Twitter made these projects possible. We then talked about how many tools built for standard English perform very poorly on AAE, and why collecting dialect-specific data is important. For the rest of the conversation, we discussed the issues involved in scraping data from social media, including ethical considerations and the biases that the data comes with.

    Brendan O’Connor is an Assistant Professor at the University of Massachusetts, Amherst.

    Warning: This episode contains explicit language (one swear word).

    • 43 min
    102 - Biomedical NLP research at the National Institute of Health with Dina Demner-Fushman

    102 - Biomedical NLP research at the National Institute of Health with Dina Demner-Fushman

    What exciting NLP research problems are involved in processing biomedical and clinical data? In this episode, we spoke with Dina Demner-Fushman, who leads NLP and IR research at the Lister Hill National Center for Biomedical Communications, part of the National Library of Medicine. We talked about processing biomedical scientific literature, understanding clinical notes, and answering consumer health questions, and the challenges involved in each of these applications. Dina listed some specific tasks and relevant data sources for NLP researchers interested in such applications, and concluded with some pointers to getting started in this field.

    • 36 min
    101 - The lottery ticket hypothesis, with Jonathan Frankle

    101 - The lottery ticket hypothesis, with Jonathan Frankle

    In this episode, Jonathan Frankle describes the lottery ticket hypothesis, a popular explanation of how over-parameterization helps in training neural networks. We discuss pruning methods used to uncover subnetworks (winning tickets) which were initialized in a particularly effective way. We also discuss patterns observed in pruned networks, stability of networks pruned at different time steps and transferring uncovered subnetworks across tasks, among other topics.

    A recent paper on the topic by Frankle and Carbin, ICLR 2019: https://arxiv.org/abs/1803.03635

    Jonathan Frankle’s homepage: http://www.jfrankle.com/

    • 41 min

Top de podcasts em Ciência

Outros ouvintes também assinaram