Sasha Rush on NLP research, engineering, and education CS224U

Coding puzzles, practices, and education, structured prediction, the culture of Hugging Face, large models, and the energy of New York.

Transcript: https://web.stanford.edu/class/cs224u/podcast/rush/


Sasha's website
Sasha on Twitter
Sasha on the Humans of AI podcast
Sasha on The Thesis Review Podcast with Sean Welleck
Sasha on the Talking Machines Podcast
Sasha interviewed by Sayak Paul
Hugging Face
PyTorch
The Annotated Transformer
The Annotated Alice
The Annotated S4
Sasha and Dan Oneață's declarative graphics library Chalk
Drawing Big Ben in Chalk
OpenNMT
Ken Shan
Blog post by Ken and Dylan Thurston
Edward Z. Yang
Stuart Shieber
Literate programming
Soumith Chintala
Lua Torch
TensorFlow
Graham Neubig
Chris Dyer
DyNet
JAX
jax.vmap
Matt Johnson
Finale Doshi-Velez, whose undergrad ML course inspired and informed Sasha's
Tensor Puzzles
GPU Puzzles
A tweet that Chris added to his CV
Adam Paszke
Dougal MacLaurin
Dex
Named Tensor notation
Named Tensors in PyTorch
TorchDim
Mini Torch
Torch-Struct
Sarah Hooker's paper 'The hardware lottery'
Jacob Andreas
Kevin Ellis
Hugging Face transformers library
Hugging Face datasets library
Hugging Face diffusers library
Hugging Face evaluate library
scikit-learn
Big Science blog
BLOOM
The Technology Behind BLOOM Training
CRFM
Eleuther
T0 and PromptSource
Washington Post: Big Tech builds AI with bad data. So scientists sought better data
The bet: Is Attention All You Need?
Democratizing access to large-scale language models with OPT-175B
Epic OPT-175 Logbook
Google's PaLM
United's shares plunge 76% on bogus bankruptcy report
Imagen
Albert Gu
Bell Labs