RoboPapers

Chris Paxton and Michael Cho
  • 5.0 (2)
  • Technology
  • Updated Weekly

Chris Paxton & Michael Cho geek out over robotic papers with paper authors. robopapers.substack.com

  1. 17h ago

    Ep#96: EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstration

    How can roboticists scale human manipulation data in the wild? EgoHumanoid builds a comprehensive pipeline which unlocks mobile manipulation training for robots — meaning that you can walk around and collect data for your robot anywhere you go. There are two key components to their recipe: view alignment, to reduce mismatch between human and robot camera views, and action alignment, to make sure human actions can be converted into robot actions. This makes it easier than ever to train robot policies and deploy them in previously-unseen environments. Modi Shi joins us to tell us the details. Watch Episode #96 of RoboPapers with Michael Cho and Jiafei Duan today! Abstract Human demonstrations offer rich environmental diversity and scale naturally, making them an appealing alternative to robot teleoperation. While this paradigm has advanced robot-arm manipulation, its potential for the more challenging, data-hungry problem of humanoid loco-manipulation remains largely unexplored. We present EgoHumanoid, the first framework to co-train a vision-language-action policy using abundant egocentric human demonstrations together with a limited amount of robot data, enabling humanoids to perform loco-manipulation across diverse real-world environments. To bridge the embodiment gap between humans and robots, including discrepancies in physical morphology and viewpoint, we introduce a systematic alignment pipeline spanning from hardware design to data processing. A portable system for scalable human data collection is developed, and we establish practical collection protocols to improve transferability. At the core of our human-to-humanoid alignment pipeline lies two key components. The view alignment reduces visual domain discrepancies caused by camera height and perspective variation. The action alignment maps human motions into a unified, kinematically feasible action space for humanoid control. Extensive real-world experiments demonstrate that incorporating robot-free egocentric data significantly outperforms robot-only baselines by 51\%, particularly in unseen environments. Our analysis further reveals which behaviors transfer effectively and the potential for scaling human data. Learn More Project page: https://opendrivelab.com/EgoHumanoid/ ArXiV: https://arxiv.org/abs/2602.10106 Github: https://github.com/OpenDriveLab/EgoHumanoid This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#96: EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstration

  2. Aug 5

    Ep#95: Action-to-Action Flow Matching

    Diffusion Policy was one of the big breakthroughs that has enabled an explosion in real-world robot learning. However, it’s always had a weakness, which is that it works by computing a final action trajectory from random noise, which leads to high latency when predicting a final action sequence. Instead, why not initialize the search based on previous actions? This allows for incredibly fast policy inference and in many cases improved generalization, generating high-quality predictions with sub-ms latency. Jindou Jia and Jianfei Yang join us to explain. Learn more on Episode 95 of RoboPapers, with Michael Cho and Chris Paxton! Abstract Diffusion-based policies have recently achieved remarkable success in robotics by formulating action prediction as a conditional denoising process. However, the standard practice of sampling from random Gaussian noise often requires multiple iterative steps to produce clean actions, leading to high inference latency that incurs a major bottleneck for real-time control. In this paper, we challenge the necessity of uninformed noise sampling and propose Action-to-Action flow matching (A2A), a novel policy paradigm that shifts from random sampling to initialization informed by the previous action. Unlike existing methods that treat proprioceptive action feedback as static conditions, A2A leverages historical proprioceptive sequences, embedding them into a high-dimensional latent space as the starting point for action generation. This design bypasses costly iterative denoising while effectively capturing the robot's physical dynamics and temporal continuity. Extensive experiments demonstrate that A2A exhibits high training efficiency, fast inference speed, and improved generalization. Notably, A2A enables high-quality action generation in as few as a single inference step (0.56 ms latency), and exhibits superior robustness to visual perturbations and enhanced generalization to unseen configurations. Lastly, we also extend A2A to video generation, demonstrating its broader versatility in temporal modeling. Learn More Project Page: https://jingliangli.com/A2A_Flow_Matching/ ArXiV: https://arxiv.org/abs/2602.07322 Github: https://github.com/JIAjindou/A2A_Flow_Matching This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#95: Action-to-Action Flow Matching

  3. Aug 3

    Ep#94: Learning a Thousand Tasks in a Day

    For general-purpose robots to be useful, we must be able to teach them new tasks quickly and repurpose them to new roles. But most techniques for teaching robots — even from human demonstrations — take a great deal of new data. But it turns out that it is possible, in part through decomposing tasks into component subtasks, that it’s possible to learn a robot skills quickly. Kamil Dreczkowski, Pietro Vitiello and Edward Johns join us to talk about their extensive study on how to teach robots new skills efficiently, while also generalizing to novel object instances, using a combination of task decomposition and retrieval, work that was published in Science Robotics. Watch episode #94 of RoboPapers now, with Michael Cho and Jiafei Duan, to learn more! Abstract Humans are remarkably efficient at learning tasks from demonstrations, but today's imitation learning methods for robot manipulation often require hundreds or thousands of demonstrations per task. We investigate two fundamental priors for improving learning efficiency: decomposing manipulation trajectories into sequential alignment and interaction phases, and retrieval-based generalisation. Through 3,450 real-world rollouts, we systematically study this decomposition. We compare different design choices for the alignment and interaction phases, and examine generalisation and scaling trends relative to today's dominant paradigm of behavioural cloning with a single-phase monolithic policy. In the few-demonstrations-per-task regime (this https URL. Learn More Project Page: https://www.robot-learning.uk/learning-1000-tasks ArXiV: https://arxiv.org/abs/2511.10110 Science Paper: https://www.science.org/doi/abs/10.1126/scirobotics.adv7594 Github: https://github.com/kamil-dreczkowski/learning_thousand_tasks This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#94: Learning a Thousand Tasks in a Day

  4. Jul 29

    Ep#93: CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy

    Many tasks cannot be completed by one robot alone. But coordinating multiple robots performing complex manipulation tasks is very difficult. Many solutions rely on complicated centralized control, which tends towards bespoke methods that do not scale well with team size. Enter CHORUS by Ria Doshi, Jeanette Bohg, and team. They train a single VLA to control multiple, diverse robots, given only local observations and identifying information. This allows robots to collaborate on tasks like using a tape measure, handing over a book, or lifting a laundry basket. To learn more, watch Episode #93 of Robopapers now, with Michael Cho, Jiafei Duan, and Chris Paxton! Abstract Multi-robot collaboration allows robots to efficiently take on a wide range of tasks, from moving a couch through a doorway to assembling structures on a construction site. However, achieving such coordination in mobile multi-robot settings remains challenging: centralized methods conditioned on the combined observations of a team scale poorly with team size, and decentralized methods that train one policy per robot often require explicit alignment procedures or information sharing at inference time to overcome partial observability. Our key insight is that the visuomotor priors of pretrained vision-language-action (VLA) models should enable reactive, decentralized collaboration from each robot's local observations alone, without these inference-time assumptions. We propose CHORUS, a framework that adapts a single VLA backbone to control diverse, multi-robot teams. At inference time, each robot runs an independent copy of CHORUS, conditioned only on its own observations and a robot-identifying prompt. In real-world experiments including mobile tape measurement, library book handovers, and laundry basket lifting, CHORUS achieves a 64% point improvement over decentralized, from-scratch models, improves reactivity to teammate behavior by 40% points, and outperforms centralized baselines. Together, these results show that a shared VLA backbone is capable of achieving decentralized multi-robot collaboration, without per-robot policies or inter-robot communication at inference. Learn More Project Page: https://chorus-model.github.io/ ArXiV: https://arxiv.org/abs/2606.12352 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#93: CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy

  5. Jul 25

    Ep#92: Human Universal Grasping

    How can we learn robot grasping from egocentric human video alone? General-purpose dexterous manipulation learning will require a lot of data, and yet robot data is hard to find at scale. Better leveraging human data, then, will be key to general-purpose dexterous manipulation. Kevin Wu, Irmak Guzey, and Dandan Shan join us to talk about HUG: Human Unversal Grasping. They collect a dataset of 1M frames (27.8 hours) of egocentric human grasping data, and use it to train a new flow-matching algorithm to predict hand pose. They can then predict human hand poses and retarget them to robot hands, allowing for a massive improvement on a wide variety of zero-shot robot grasping tasks in everyday scenes. To learn more, watch Episode 92 of RoboPapers with Michael Cho and Chris Paxton today! Abstract Humans can grasp objects effortlessly, whereas multi-fingered robots are far from this level of generality. We argue that the most natural source of robot grasping data is from humans, who pick up thousands of objects every day. We present HUG, a flow-matching model that generates diverse human grasps for any user-specified object in a single RGB-D image captured from a stereo camera. Using smart glasses, we first collect 1M-HUGs, an egocentric dataset of human grasps spanning 1M frames (27.8 hrs) and 6,707 object instances across 41 buildings. Next, to model the distribution of natural human grasps, our novel flow-matching model fuses RGB and depth observations to output a grasp parameterized by wrist translation, wrist rotation, and MANO hand pose. Predicted grasps can be retargeted to various robot hands, enabling zero-shot grasping in everyday scenes. To standardize evaluation, we build a new simulated benchmark, HUG-Bench, of 90 unseen objects from five geometric categories and various sizes, with metric-scale 3D meshes. We evaluate HUG in the real world on the 30-object test set of HUG-Bench across multiple stereo cameras, robot embodiments, and household environments. HUG outperforms the state-of-the-art grasping baselines by +23% and +34% on our challenging object set. Code, data, benchmark, checkpoints, and an interactive demo are released on our website: this https URL Learn More Project Page: https://grasping.io/ ArXiV: https://arxiv.org/abs/2606.17054 Github: https://github.com/KevinyWu/hug This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

  6. Jul 23

    Ep#91: RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

    Memory is one of the most important problems in robotics. Long horizon memory is key for a variety of robot manipulation problems. However, there exist no good benchmarks for understanding progress in how well generalist robot policies can understand language. Yinpei Dai and Yuejiang Liu made RoboMME as a solution: it’s a large benchmark which shows 16 different robot tasks, like counting objects or mastering timing. They show 14 different memory-augmented generalist policies across these different benchmarks. It’s an incredibly thorough and interesting result, aimed at driving forward this core robotic capability. To learn more, watch Episode 91 of RoboPapers with Michael Cho and Chris Paxton! Abstract Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipulating objects that become temporarily occluded. Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms; however, their evaluations remain confined to narrow, non-standardized settings. This limits their systematic understanding, comparison, and progress measurement. To address these challenges, we introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models in long-horizon, history-dependent scenarios. Our benchmark comprises 16 manipulation tasks constructed under a carefully designed taxonomy that evaluates temporal, spatial, object, and procedural memory. We further develop a suite of 14 memory-augmented VLA variants built on the π0.5 backbone to systematically explore different memory representations across multiple integration strategies. Experimental results show that the effectiveness of memory representations is highly task-dependent, with each design offering distinct advantages and limitations across different tasks. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#91: RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

  7. Jul 15

    Ep#90: From Capable Controllers to Deployable Humanoid Systems

    We want humanoid robots to be able to perform complex, long-horizon tasks in the real world — putting away the groceries or cleaning a room, for example. This requires diverse loco-manipulation skills, which can be easily parameterized to handle object affordances and interact with the world around it safely. In HANDOFF, Lizhi Yang proposes a 10-D learned whole body controller, which can be converted to whole body actions, and which can be used by a VLM-driven agentic planner to perform complex, multi-step manipulation actions in the real world. We then discuss how it’s possible to deploy such controllers in the real world, how to make them safe around people via controlled barrier functions and safety functions. This enables humanoids which can move safely through dynamic, crowded environments in the real world. Learn more in Episode 90 of RoboPapers, with Michael Cho and Chris Paxton. Abstract For a humanoid robot to be deployed in the real world, the choice of command space (i.e., the interface between task planning and whole-body control) is crucial. Existing whole-body controllers typically demand dense kinematic or spatial references that planners struggle to synthesize from task semantics. We instead propose a compact, explicit interface that is intuitive, general, modular, and expressive enough for diverse loco-manipulation skills. To this end, we introduce HANDOFF, a single humanoid whole-body controller that follows this interface and is distilled via multi-teacher KL distillation under a context-conditioned gating scheme into a mixture-of-experts student from three complementary specialists: whole-body motion tracking with safety-filtered data, locomotion, and fall-recovery. On the Unitree G1, HANDOFF matches state-of-the-art velocity tracking and offers one of the largest robust manipulation workspaces. We further demonstrate hardware feasibility through multiple natural-language-driven task roll-outs, powered by a VLM-driven agentic planner with no task-specific data or controller fine-tuning. Learn More Project page for HANDOFF: https://lzyang2000.github.io/HANDOFF/ Code: https://github.com/lzyang2000/HANDOFF ArXiV: https://arxiv.org/abs/2606.06493 Safe-SAGE on ArXiV: https://arxiv.org/abs/2603.05497 SHIELD on ArXiV: https://arxiv.org/abs/2505.11494 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#90: From Capable Controllers to Deployable Humanoid Systems

  8. Jul 8

    Ep#89: Contact Grounded Policy

    Contact-rich manipulation is still very challenging for robotics. Problems like opening a jar, or in-hand reorientation of an object, require making repeated contact with different parts of a robot’s hand, and this is hard to do with pure vision. Instead, research is moving towards using tactile sensors in combination with visual policies. But what’s the best way to learn how to handle multi-point contact? Zhengtong Xu and Yeping Wang tell us about their new work Contact-Grounded Policy (CGP). CGP predicts future robot state and tactile feedback, and predicts this into actions for a compliant robot controller so that a four- or five-finger robot hand can perform complex tasks involving precise manipulation, delicate grasping, and tool use. To learn more, watch Episode #89 of RoboPapers, with Chris Paxton and Jiafei Duan. Abstract Contact-rich dexterous manipulation with multi-finger hands remains an open challenge in robotics because task success depends on multi-point contacts that continuously evolve and are highly sensitive to object geometry, frictional transitions, and slip. Recently, tactile-informed manipulation policies have shown promise. However, most use tactile signals as additional observations rather than modeling contact state or how their action outputs interact with low-level controller dynamics. We present Contact-Grounded Policy (CGP), a visuotactile policy that grounds multi-point contacts by predicting coupled trajectories of actual robot state and tactile feedback, and using a learned contact-consistency mapping to convert these predictions into executable target robot states for a compliance controller. CGP consists of two components: (i) a conditional diffusion model that forecasts future robot state and tactile feedback in a compressed latent space, and (ii) a learned contact-consistency mapping that converts the predicted robot state-tactile pair into executable targets for a compliance controller, enabling it to realize the intended contacts. We evaluate CGP using a physical four-finger Allegro V5 hand with Digit360 fingertip tactile sensors, and a simulated five-finger Tesollo DG-5F hand with dense whole-hand tactile arrays. Across a range of dexterous tasks including in-hand manipulation, delicate grasping, and tool use, CGP outperforms visuomotor and visuotactile diffusion-policy baselines. Learn More Project page: https://contact-grounded-policy.github.io/ ArXiV: https://arxiv.org/abs/2603.05687 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

    Ep#89: Contact Grounded Policy
See All (96)

Ratings & Reviews

5
out of 5
2 Ratings

About

Chris Paxton & Michael Cho geek out over robotic papers with paper authors. robopapers.substack.com

Information

  • Creator
    Chris Paxton and Michael Cho
  • Years Active
    2025 - 2026
  • Episodes
    96
  • Rating
    Clean
  • Copyright
    © Chris Paxton
  • Show Website
    RoboPapers

You Might Also Like