1 hr 15 min
Pete Florence: Dense Visual Representations, NeRFs, and LLMs for Robotics The Gradient: Perspectives on AI
-
- Technology
In episode 54 of The Gradient Podcast, Andrey Kurenkov speaks with Pete Florence.
Note: this was recorded 2 months ago. Andrey should be getting back to putting out some episodes next year.
Pete Florence is a Research Scientist at Google Research on the Robotics at Google team inside Brain Team in Google Research. His research focuses on topics in robotics, computer vision, and natural language -- including 3D learning, self-supervised learning, and policy learning in robotics. Before Google, he finished his PhD in Computer Science at MIT with Russ Tedrake.
Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter
Outline:
* (00:00:00) Intro
* (00:01:16) Start in AI
* (00:04:15) PhD Work with Quadcopters
* (00:08:40) Dense Visual Representations
* (00:22:00) NeRFs for Robotics
* (00:39:00) Language Models for Robotics
* (00:57:00) Talking to Robots in Real Time
* (01:07:00) Limitations
* (01:14:00) Outro
Papers discussed:
* Aggressive quadrotor flight through cluttered environments using mixed integer programming
* Integrated perception and control at high speed: Evaluating collision avoidance maneuvers without maps
* High-speed autonomous obstacle avoidance with pushbroom stereo
* Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation. (Best Paper Award, CoRL 2018)
* Self-Supervised Correspondence in Visuomotor Policy Learning (Best Paper Award, RA-L 2020 )
* iNeRF: Inverting Neural Radiance Fields for Pose Estimation.
* NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields.
* Reinforcement Learning with Neural Radiance Fields
* Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language.
* Inner Monologue: Embodied Reasoning through Planning with Language Models
* Code as Policies: Language Model Programs for Embodied Control
Get full access to The Gradient at thegradientpub.substack.com/subscribe
In episode 54 of The Gradient Podcast, Andrey Kurenkov speaks with Pete Florence.
Note: this was recorded 2 months ago. Andrey should be getting back to putting out some episodes next year.
Pete Florence is a Research Scientist at Google Research on the Robotics at Google team inside Brain Team in Google Research. His research focuses on topics in robotics, computer vision, and natural language -- including 3D learning, self-supervised learning, and policy learning in robotics. Before Google, he finished his PhD in Computer Science at MIT with Russ Tedrake.
Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter
Outline:
* (00:00:00) Intro
* (00:01:16) Start in AI
* (00:04:15) PhD Work with Quadcopters
* (00:08:40) Dense Visual Representations
* (00:22:00) NeRFs for Robotics
* (00:39:00) Language Models for Robotics
* (00:57:00) Talking to Robots in Real Time
* (01:07:00) Limitations
* (01:14:00) Outro
Papers discussed:
* Aggressive quadrotor flight through cluttered environments using mixed integer programming
* Integrated perception and control at high speed: Evaluating collision avoidance maneuvers without maps
* High-speed autonomous obstacle avoidance with pushbroom stereo
* Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation. (Best Paper Award, CoRL 2018)
* Self-Supervised Correspondence in Visuomotor Policy Learning (Best Paper Award, RA-L 2020 )
* iNeRF: Inverting Neural Radiance Fields for Pose Estimation.
* NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields.
* Reinforcement Learning with Neural Radiance Fields
* Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language.
* Inner Monologue: Embodied Reasoning through Planning with Language Models
* Code as Policies: Language Model Programs for Embodied Control
Get full access to The Gradient at thegradientpub.substack.com/subscribe
1 hr 15 min