Advances in Human-Robot Communication and Task Planning

The field of human-robot collaboration is moving towards more effective communication and task planning, with a focus on leveraging speech prosody, uncertainty-informed action selection, and large language models to improve robot understanding and execution of spoken language instructions. Researchers are exploring innovative approaches to address the challenges of ambiguous human instructions, hidden or unknown object locations, and open-vocabulary object types. Notable papers include:

  • Tru-POMDP, which achieves higher success rates and stronger robustness to ambiguity and occlusion in task planning under uncertainty.
  • Efficient Manipulation-Enhanced Semantic Mapping, which highly reduces object displacement and drops while achieving a 95% reduction in planning time compared to the state-of-the-art.

Sources

Enhancing Speech Instruction Understanding and Disambiguation in Robotics via Speech Prosody

Efficient Manipulation-Enhanced Semantic Mapping With Uncertainty-Informed Action Selection

Tru-POMDP: Task Planning Under Uncertainty via Tree of Hypotheses and Open-Ended POMDPs

Understanding Physical Properties of Unseen Deformable Objects by Leveraging Large Language Models and Robot Actions

AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment

Built with on top of