RoboPapers

Chris Paxton and Michael Cho
  • 5.0 (3)
  • Technology
  • Updated Weekly

Chris Paxton & Michael Cho geek out over robotic papers with paper authors. robopapers.substack.com

  1. 2d ago

    Ep#103: Freeform Preference Learning for Robotic Manipulation

    Specifying reward functions for robots is one of the hardest things about reinforcement learning. Robot rewards often need to be very detailed; metrics like progress can be ill-defined and hard to estimate. This leads to most robot learning defaulting to sparse rewards or simple preference learning. But naive preference learning (having human annotators choose one trajectory over another) is an easy solution, but obscures a lot of the signal in complex tasks and can make learning a lot less efficient. Marcel Torné, Anubha Mahajan, and Abhijnya Bhat join us to talk about their solution: freeform preference learning, which lets annotators define natural-language axes to compare trajectories over. This improves real-world performance on long-horizon manipulation tasks over sparse rewards and simple binary preference learning. Watch Epsiode 103 of RoboPapers, with Michael Cho and Jiafei Duan, today to learn more! Abstract Reward design remains a central bottleneck for autonomous robot policy improvement, especially in long-horizon manipulation tasks where sparse success labels provide too little signal and binary preferences collapse many competing notions of quality into one ambiguous signal. We introduce Freeform Preference Learning (FPL), a method for learning robot policies from freeform human preferences. Rather than asking annotators which of two trajectories is better overall, FPL lets them define natural-language preference axes, such as speed, safety, quality of placement, or carefulness, and provide pairwise preferences along each axis. These annotations are used to learn a language-conditioned reward model that maps a trajectory and preference label to an axis-specific reward. We use this model to train a reward-conditioned policy that optimizes across the multiple human-specified dimensions. Across four real-world and two simulated long-horizon manipulation tasks, FPL improves over sparse-reward and binary-preference methods by 38 percentage points. Beyond improved performance, FPL learns dense progress signals without explicit subtask segmentation, shows compositionality of behavior not present in the data, and allows users to steer the policy towards different behaviors at test time without retraining. Blog post with videos available at this https URL Learn More Project page: https://freeform-pl.github.io/fpl.website/ ArXiV: https://arxiv.org/abs/2606.32027 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#103: Freeform Preference Learning for Robotic Manipulation

  2. 3d ago

    Ep#102: EgoSuite-Open100K from Lightwheel

    Robotics has a data problem, and egocentric data is a compelling way to solve it because egocentric data is (1) scalable — it’s cheap to collect — and (2) easily captures the breadth and diversity of real human tasks, while (3) being true to real physics. Lightwheel recently released a hundred thousand hours of high-quality egocentric video data, spread across a wide range of human tasks from cleaning and tidying to assembly and construction to retail tasks. Here to tell us about this tremendous effort is Steve Xie, CEO and co-founder of Lightwheel. The data is also available on HuggingFace right now. Watch Episode 102 of RoboPapers, with Michael Cho and Jiafei Duan, to learn more! Learn More Project page: https://egosuite100k.lightwheel.ai/ Blog post: https://huggingface.co/blog/LightwheelAI/egosuite-open100k Data on HuggingFace: https://huggingface.co/collections/LightwheelAI/egosuite-open100k This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#102: EgoSuite-Open100K from Lightwheel

  3. Sep 4

    Ep#101: Coupled Local and Global World Models for Efficient First Order RL

    Reinforcement learning has unlocked tremendous gains in broader AI and humanoid robot whole body control, but still has yet to deliver strong results for robot manipulation due to the difficulty of capturing tasks in an accurate simulation; one way to fix this is to create complex environments with learned world models. But learning fully within world models is often intractable as powerful world models are computationally too expensive. Instead, Joseph Amigo and Rooholla Khorrambahkt propose to break the problem down: into a large-scale world model that can generate forward trajectories, and a lightweight, low-dimensional latent-space model which can approximate local dynamics of a problem, without needing to back-propagate through a heavy global model. What this means: you can do reinforcement learning in this coupled local-global world model, and learn contact rich skills for a robot. To learn more, watch Episode #101 of RoboPapers, with Michael Cho and Jiafei Duan! Abstract World models offer a promising avenue for capturing complex environment dynamics where simulators face challenges. However, large-scale world models required for complex real-world settings are computationally expensive to adopt in popular RL approaches. We introduce a novel first-order RL method that enables policy training via a decoupled first-order gradient (FoG): a large-scale world model generates accurate forward trajectories while a lightweight latent-space surrogate approximates its local dynamics for efficient gradient computation. This coupled local-and-global formulation allows high-fidelity forward dynamics alongside the computationally efficient differentiation needed for model-based RL. Across a range of real-world robotic tasks we demonstrate tractable RL and zero-shot deployment, with significantly better sample efficiency than PPO on a canonical real-world Push-T benchmark and similar gains on more complex ego-centric manipulation and grasping. Learn More Project page: https://coupled-global-local-wm-rl.pages.dev/ ArXiV: https://arxiv.org/abs/2602.06219 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#101: Coupled Local and Global World Models for Efficient First Order RL

  4. Sep 2

    Ep#100: 100 Episodes of Embodied AI

    Embodied AI is exploding, and it’s been amazing to learn about it with you all. We have now recorded over a hundred discussions with the best researchers in the world on embodied artificial intelligence and robotics, covering topics like scaling robot learning, humanoid whole body control, tactile sensors, sim-to-real, and so much more. For our hundredth episode, we brought on our newest co-host, Ruijie He, to discuss what we’ve seen and learned in the first hundred episodes. Which were our most-watched episodes? Which institution shows up the most? Who was our most frequent guest? What did we learn? Find out on Episode 100 of RoboPapers, with Michael Cho, Chris Paxton, Jiafei Duan, and Ruijie He! Learn More https://robopapers100.com/ This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#100: 100 Episodes of Embodied AI

  5. Aug 18

    Ep#99: DYNA-2: A 1 Million Hour Scaling Law for World-Action Models

    Excitement about world-action models and robot learning has never been higher — they promise a way to use human egocentric data to train massive robotics models which can provide the “GPT” moment for robotics and unlock general-purpose embodied intelligence. And yet there’s been little concrete demonstration of scaling in robot learning. Dyna Robotics aims to change that, with an in-depth look at how scaling works as they approach 1 million hours of training data. Jason Ma, Tianyu Li, Anh-Quan Pham, and Chet Bhateja joined us to tell us more. They show that as the amount of data they use in pretraining increased, they saw predictable, statistically significant gains on accuracy metrics on held-out data (data not seen during training). They go on to talk about what they learned, and show how this can be applied to many different problems. Watch Episode #99 of RoboPapers, with Michael Cho, Chris Paxton, and Jiafei Duan today to learn more! Learn More Read the blog post: This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

  6. Aug 14

    Ep#97: Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

    Representing actions for robot hands is actually very challenging. Robot hands have different sizes and shapes, and different degrees of freedom. Representing their actions purely as joint states — as we naively would — leads to less transfer across the wide variety of available hands. Instead, could we design a universal hand action space to allow robots to learn cross-embodiment dexterous manipulation more easily? Luis Felipe Casas and Yu Xiang come on to tell us about UHAS: the Unified Hand Action Space, a sphere-based action representation which enables cross-ebmodiment learning of dexterous manipulation skills. Watch Episode #97 of RoboPapers, with Chris Paxton and Jiafei Duan, today to learn more! Abstract Robot manipulation policies are typically tied to specific robotic hand embodiments, limiting the transfer of learned behaviors across platforms with different kinematic structures. In this work, we propose the Unified Hand Action Space (UHAS), a sphere-based unified action representation for cross-embodiment dexterous manipulation. UHAS represents robotic hand actions as geometric deformations of a canonical sphere and uses a Cascade Inverse Kinematics (CIK) algorithm to map the shared representation to embodiment-specific joint configurations. Using reinforcement learning, we train dexterous manipulation policies directly in the proposed action space for in-hand cube reorientation tasks. We evaluate our method in both simulation and real-world experiments across multiple robotic hands, including the Allegro Hand, LEAP Hand, Shadow Hand, and MANO Human Hand. Experimental results demonstrate effective dexterous manipulation, zero-shot transfer to unseen hands, rapid finetuning across embodiments, and successful real-world deployment. Our experiments show that the proposed UHAS representation enables stable dexterous control and cross-embodiment policy transfer across robotic hands. Learn More ArXiV: https://arxiv.org/abs/2607.03570 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#97: Cross-Embodiment Robot Manipulation via a Unified Hand Action Space

  7. Aug 12

    Ep#96: EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstration

    How can roboticists scale human manipulation data in the wild? EgoHumanoid builds a comprehensive pipeline which unlocks mobile manipulation training for robots — meaning that you can walk around and collect data for your robot anywhere you go. There are two key components to their recipe: view alignment, to reduce mismatch between human and robot camera views, and action alignment, to make sure human actions can be converted into robot actions. This makes it easier than ever to train robot policies and deploy them in previously-unseen environments. Modi Shi joins us to tell us the details. Watch Episode #96 of RoboPapers with Michael Cho and Jiafei Duan today! Abstract Human demonstrations offer rich environmental diversity and scale naturally, making them an appealing alternative to robot teleoperation. While this paradigm has advanced robot-arm manipulation, its potential for the more challenging, data-hungry problem of humanoid loco-manipulation remains largely unexplored. We present EgoHumanoid, the first framework to co-train a vision-language-action policy using abundant egocentric human demonstrations together with a limited amount of robot data, enabling humanoids to perform loco-manipulation across diverse real-world environments. To bridge the embodiment gap between humans and robots, including discrepancies in physical morphology and viewpoint, we introduce a systematic alignment pipeline spanning from hardware design to data processing. A portable system for scalable human data collection is developed, and we establish practical collection protocols to improve transferability. At the core of our human-to-humanoid alignment pipeline lies two key components. The view alignment reduces visual domain discrepancies caused by camera height and perspective variation. The action alignment maps human motions into a unified, kinematically feasible action space for humanoid control. Extensive real-world experiments demonstrate that incorporating robot-free egocentric data significantly outperforms robot-only baselines by 51\%, particularly in unseen environments. Our analysis further reveals which behaviors transfer effectively and the potential for scaling human data. Learn More Project page: https://opendrivelab.com/EgoHumanoid/ ArXiV: https://arxiv.org/abs/2602.10106 Github: https://github.com/OpenDriveLab/EgoHumanoid This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#96: EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstration

  8. Aug 5

    Ep#95: Action-to-Action Flow Matching

    Diffusion Policy was one of the big breakthroughs that has enabled an explosion in real-world robot learning. However, it’s always had a weakness, which is that it works by computing a final action trajectory from random noise, which leads to high latency when predicting a final action sequence. Instead, why not initialize the search based on previous actions? This allows for incredibly fast policy inference and in many cases improved generalization, generating high-quality predictions with sub-ms latency. Jindou Jia and Jianfei Yang join us to explain. Learn more on Episode 95 of RoboPapers, with Michael Cho and Chris Paxton! Abstract Diffusion-based policies have recently achieved remarkable success in robotics by formulating action prediction as a conditional denoising process. However, the standard practice of sampling from random Gaussian noise often requires multiple iterative steps to produce clean actions, leading to high inference latency that incurs a major bottleneck for real-time control. In this paper, we challenge the necessity of uninformed noise sampling and propose Action-to-Action flow matching (A2A), a novel policy paradigm that shifts from random sampling to initialization informed by the previous action. Unlike existing methods that treat proprioceptive action feedback as static conditions, A2A leverages historical proprioceptive sequences, embedding them into a high-dimensional latent space as the starting point for action generation. This design bypasses costly iterative denoising while effectively capturing the robot's physical dynamics and temporal continuity. Extensive experiments demonstrate that A2A exhibits high training efficiency, fast inference speed, and improved generalization. Notably, A2A enables high-quality action generation in as few as a single inference step (0.56 ms latency), and exhibits superior robustness to visual perturbations and enhanced generalization to unseen configurations. Lastly, we also extend A2A to video generation, demonstrating its broader versatility in temporal modeling. Learn More Project Page: https://jingliangli.com/A2A_Flow_Matching/ ArXiV: https://arxiv.org/abs/2602.07322 Github: https://github.com/JIAjindou/A2A_Flow_Matching This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#95: Action-to-Action Flow Matching
See All (102)

Ratings & Reviews

5
out of 5
3 Ratings

About

Chris Paxton & Michael Cho geek out over robotic papers with paper authors. robopapers.substack.com

Information

  • Creator
    Chris Paxton and Michael Cho
  • Years Active
    2025 - 2026
  • Episodes
    102
  • Rating
    Clean
  • Copyright
    © Chris Paxton
  • Show Website
    RoboPapers

You Might Also Like