DNN 7: Reinforcement Learning | Research at Waymo, University of Oxford | Shimon Whiteson Deep Neural Notebooks

In the seventh episode of Deep Neural Notebooks, I interview Shimon Whiteson.  

Shimon sir is a Computer Science Professor at the University of Oxford, where he leads the Whiteson Research Lab. He is also a Data Scientist at Waymo (formerly the Google Self Driving Car Project). His research specialises in Reinforcement Learning (RL), Cooperative Multi-Agent RL, to be precise.   

So this interview is all in the context of Reinforcement Learning. We talk about his journey  - how he started with Machine Learning & RL. I ask him about his thoughts on the state of RL - about how the field has progressed and changed since he started, about how it has become so popular in the last few years, and about the challenges being faced.  

We also talk about his research at Waymo, about recent projects from his lab, and about the scope and future of telepresence robots, one of which was developed under his guidance. We also talk about the infamous Reward Hypothesis in the context of RL and Philosophy. In the end, he also shares some advice for people starting out with RL.  

Links:  

- Shimon Whiteson: https://twitter.com/shimon8282 

- Whiteson Research Lab (WhiRL): http://whirl.cs.ox.ac.uk/ 

- Teresa Robot: https://whirl.cs.ox.ac.uk/teresa/ 

- RL workshop at Machine Learning Summer School, Moscow: https://www.youtube.com/watch?v=RAw0Chs7QKA 

- The Reward Hypothesis: http://incompleteideas.net/rlai.cs.ualberta.ca/RLAI/rewardhypothesis.html



Timestamps:

03:42 Beginnings in Computer Science06:13 Beginnings in ML

07:15 PhD at UT Austin

10:40 Intersection of Neuroevolution and RL

14:10 Research directions since PhD

16:35 State of RL

20:33 Simulation for RL

22:07 Research at Waymo

25:30 Multi-agent RL

33:25 Recent projects at WhiRL

41:30 Teresa project and Telepresence Robots

48:08 Bottlenecks for RL and Robotics

49:45 End-goal for RL, Human-level Intelligence

53:45 What do you find most fascinating about your research?

55:38 RL & Philosophy

1:01:20 Keeping up with latest research

1:03:28 Advice for beginners



Podcast links :

Youtube: https://youtu.be/bbrYZDgPI9M

Apple Podcasts:  https://apple.co/2TLUZ0y

Google Podcasts: https://bit.ly/2TIyvh6

Spotify:  https://open.spotify.com/episode/3936aEvSwsIhfwQfURmDb9

Anchor: https://bit.ly/3gpMi65



Connect:  

Twitter: https://twitter.com/mkulkhanna 

Website: https://mukulkhanna.co 

LinkedIn: https://linkedin.com/in/mukulkhanna/