Advances in Robot Learning from Demonstrations

The field of robot learning is moving towards addressing the challenges of learning from imbalanced and limited datasets. Researchers are exploring methods to rebalance datasets, reduce shortcut learning, and improve generalization capabilities of generalist robot policies. Novel approaches include analogical reasoning, variational bottlenecks, and physical autoregressive models. Notable papers include:

  • Towards Balanced Behavior Cloning from Imbalanced Datasets, which introduces a meta-gradient rebalancing algorithm to address the limitations of existing approaches.
  • AR-VRM: Imitating Human Motions for Visual Robot Manipulation with Analogical Reasoning, which proposes a keypoint Vision-Language Model pretraining scheme to learn human action knowledge.
  • Masquerade: Learning from In-the-wild Human Videos using Data-Editing, which edits in-the-wild egocentric human videos to bridge the visual embodiment gap between humans and robots.

Sources

Towards Balanced Behavior Cloning from Imbalanced Datasets

Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation

AR-VRM: Imitating Human Motions for Visual Robot Manipulation with Analogical Reasoning

AgentWorld: An Interactive Simulation Platform for Scene Construction and Mobile Robotic Manipulation

Boosting Action-Information via a Variational Bottleneck on Unlabelled Robot Videos

BEAVR: Bimanual, multi-Embodiment, Accessible, Virtual Reality Teleoperation System for Robots

Physical Autoregressive Model for Robotic Manipulation without Action Pretraining

Toward Human-Robot Teaming: Learning Handover Behaviors from 3D Scenes

Masquerade: Learning from In-the-wild Human Videos using Data-Editing

Built with on top of