Imitation Learning for Robot Manipulation

The field of robot manipulation is moving towards more efficient and robust imitation learning methods. Researchers are exploring ways to leverage simulators, online imitation learning, and force information to improve the performance of visuomotor policies. The use of digital twins, synthetic data, and compliant flow matching policies are also being investigated to address the challenges of contact-rich tasks and the sim-to-real gap. Notable papers in this area include:

  • A Recipe for Efficient Sim-to-Real Transfer in Manipulation with Online Imitation-Pretrained World Models, which proposes a sim-to-real framework that combines online imitation pretraining with offline finetuning.
  • Flow with the Force Field: Learning 3D Compliant Flow Matching Policies from Force and Demonstration-Guided Simulation Data, which introduces a framework for generating force-informed data in simulation and improves the performance of visuomotor policies.
  • EmbodiSwap for Zero-Shot Robot Imitation Learning, which presents a method for producing photorealistic synthetic robot overlays over human video and achieves an 82% success rate in real-world tests.
  • Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators, which provides a framework to augment large-scale simulation with relatively small-scale real-world testing and saves over 20-25% of hardware evaluation effort.
  • MobRT: A Digital Twin-Based Framework for Scalable Learning in Mobile Manipulation, which presents a digital twin-based framework designed to simulate complex, whole-body tasks and improves policy generalization and performance.

Sources

A Recipe for Efficient Sim-to-Real Transfer in Manipulation with Online Imitation-Pretrained World Models

Flow with the Force Field: Learning 3D Compliant Flow Matching Policies from Force and Demonstration-Guided Simulation Data

EmbodiSwap for Zero-Shot Robot Imitation Learning

Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators

MobRT: A Digital Twin-Based Framework for Scalable Learning in Mobile Manipulation

Built with on top of