The field of agent evaluation and goal recognition is moving towards more nuanced and flexible methods for assessing agent performance. Rather than relying on coarse task success metrics, researchers are exploring ways to induce fine-grained metrics from open-ended feedback, enabling more effective evaluation and improvement of language agents. Additionally, there is a growing focus on detecting and measuring goal drift in autonomous agents, which is crucial for safe operation. Benchmarking frameworks are also being developed to evaluate the capabilities of GUI-navigation AI agents. Notable papers include:

• AutoLibra, which proposes a framework for agent evaluation that transforms open-ended human feedback into concrete metrics.
• GRAML, which introduces a metric learning approach to goal recognition, enabling quick adaptation to new goals.