58 min
AIAP: On DeepMind, AI Safety, and Recursive Reward Modeling with Jan Leike Future of Life Institute Podcast
-
- Technology
Jan Leike is a senior research scientist who leads the agent alignment team at DeepMind. His is one of three teams within their technical AGI group; each team focuses on different aspects of ensuring advanced AI systems are aligned and beneficial. Jan's journey in the field of AI has taken him from a PhD on a theoretical reinforcement learning agent called AIXI to empirical AI safety research focused on recursive reward modeling. This conversation explores his movement from theoretical to empirical AI safety research — why empirical safety research is important and how this has lead him to his work on recursive reward modeling. We also discuss research directions he's optimistic will lead to safely scalable systems, more facets of his own thinking, and other work being done at DeepMind.
Topics discussed in this episode include:
-Theoretical and empirical AI safety research
-Jan's and DeepMind's approaches to AI safety
-Jan's work and thoughts on recursive reward modeling
-AI safety benchmarking at DeepMind
-The potential modularity of AGI
-Comments on the cultural and intellectual differences between the AI safety and mainstream AI communities
-Joining the DeepMind safety team
You can find the page and transcript for this podcast here: https://futureoflife.org/2019/12/16/ai-alignment-podcast-on-deepmind-ai-safety-and-recursive-reward-modeling-with-jan-leike/
Timestamps:
0:00 intro
2:15 Jan's intellectual journey in computer science to AI safety
7:35 Transitioning from theoretical to empirical research
11:25 Jan's and DeepMind's approach to AI safety
17:23 Recursive reward modeling
29:26 Experimenting with recursive reward modeling
32:42 How recursive reward modeling serves AI safety
34:55 Pessimism about recursive reward modeling
38:35 How this research direction fits in the safety landscape
42:10 Can deep reinforcement learning get us to AGI?
42:50 How modular will AGI be?
44:25 Efforts at DeepMind for AI safety benchmarking
49:30 Differences between the AI safety and mainstream AI communities
55:15 Most exciting piece of empirical safety work in the next 5 years
56:35 Joining the DeepMind safety team
Jan Leike is a senior research scientist who leads the agent alignment team at DeepMind. His is one of three teams within their technical AGI group; each team focuses on different aspects of ensuring advanced AI systems are aligned and beneficial. Jan's journey in the field of AI has taken him from a PhD on a theoretical reinforcement learning agent called AIXI to empirical AI safety research focused on recursive reward modeling. This conversation explores his movement from theoretical to empirical AI safety research — why empirical safety research is important and how this has lead him to his work on recursive reward modeling. We also discuss research directions he's optimistic will lead to safely scalable systems, more facets of his own thinking, and other work being done at DeepMind.
Topics discussed in this episode include:
-Theoretical and empirical AI safety research
-Jan's and DeepMind's approaches to AI safety
-Jan's work and thoughts on recursive reward modeling
-AI safety benchmarking at DeepMind
-The potential modularity of AGI
-Comments on the cultural and intellectual differences between the AI safety and mainstream AI communities
-Joining the DeepMind safety team
You can find the page and transcript for this podcast here: https://futureoflife.org/2019/12/16/ai-alignment-podcast-on-deepmind-ai-safety-and-recursive-reward-modeling-with-jan-leike/
Timestamps:
0:00 intro
2:15 Jan's intellectual journey in computer science to AI safety
7:35 Transitioning from theoretical to empirical research
11:25 Jan's and DeepMind's approach to AI safety
17:23 Recursive reward modeling
29:26 Experimenting with recursive reward modeling
32:42 How recursive reward modeling serves AI safety
34:55 Pessimism about recursive reward modeling
38:35 How this research direction fits in the safety landscape
42:10 Can deep reinforcement learning get us to AGI?
42:50 How modular will AGI be?
44:25 Efforts at DeepMind for AI safety benchmarking
49:30 Differences between the AI safety and mainstream AI communities
55:15 Most exciting piece of empirical safety work in the next 5 years
56:35 Joining the DeepMind safety team
58 min