Integrating Large Language Models with Classical Planning for Adaptive Robotics

The field of robotics is moving towards integrating large language models (LLMs) with classical planning to enable adaptive and goal-driven task execution in dynamic environments. This approach allows robots to interpret object affordances and interaction rules, enabling action planning and real-time adaptability. The use of LLMs provides a way to leverage commonsense knowledge and ground actions, while classical planning provides a framework for defining goals and constraints. This integration enables robots to make unfeasible tasks tractable by defining functionally equivalent goals through gradual relaxation, supporting partial achievement of the intended objective. Notable papers in this area include:

  • Context Matters! which presents an approach integrating classical planning with LLMs, leveraging their ability to extract commonsense knowledge and ground actions.
  • Grounding Language Models with Semantic Digital Twins for Robotic Planning, which introduces a novel framework that integrates Semantic Digital Twins with LLMs to enable adaptive and goal-driven robotic task execution. These papers demonstrate the potential of LLM-driven control design and planning, paving the way for advanced techniques like model predictive control and reinforcement learning.

Sources

Context Matters! Relaxing Goals with LLMs for Feasible 3D Scene Planning

Grounding Language Models with Semantic Digital Twins for Robotic Planning

AgenticControl: An Automated Control Design Framework Using Large Language Models

Adaptive Domain Modeling with Language Models: A Multi-Agent Approach to Task Planning

PSALM-V: Automating Symbolic Planning in Interactive Visual Environments with Large Language Models

Fine-Tuning and Prompt Engineering of LLMs, for the Creation of Multi-Agent AI for Addressing Sustainable Protein Production Challenges

FaSTA$^*$: Fast-Slow Toolpath Agent with Subroutine Mining for Efficient Multi-turn Image Editing

LLM-guided Chemical Process Optimization with a Multi-Agent Approach

STEP Planner: Constructing cross-hierarchical subgoal tree as an embodied long-horizon task planner

Built with on top of