The field of large language models (LLMs) is moving towards more efficient inference and decision-making methods. Researchers are exploring ways to combine the strengths of small and large models to achieve better performance while reducing costs. This includes developing frameworks for collaborative inference, cascaded decision-making, and cost-effective human-AI collaboration. Another area of focus is understanding the mechanisms underlying state-of-the-art models, such as Mamba, and improving their performance in tasks like continuous control and meta-reinforcement learning. Noteworthy papers include:
- Collaborative LLM Inference via Planning for Efficient Reasoning, which proposes a test-time collaboration framework for small and large models.
- AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes, which enables efficient knowledge transfer from teacher agents to student agents.
- Cost-Efficient Serving of LLM Agents via Test-Time Plan Caching, which reduces the cost of serving LLM-based agents by caching and reusing structured plan templates.