Efficient Inference and Decision-Making in Large Language Models

The field of large language models (LLMs) is moving towards more efficient inference and decision-making methods. Researchers are exploring ways to combine the strengths of small and large models to achieve better performance while reducing costs. This includes developing frameworks for collaborative inference, cascaded decision-making, and cost-effective human-AI collaboration. Another area of focus is understanding the mechanisms underlying state-of-the-art models, such as Mamba, and improving their performance in tasks like continuous control and meta-reinforcement learning. Noteworthy papers include:

  • Collaborative LLM Inference via Planning for Efficient Reasoning, which proposes a test-time collaboration framework for small and large models.
  • AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes, which enables efficient knowledge transfer from teacher agents to student agents.
  • Cost-Efficient Serving of LLM Agents via Test-Time Plan Caching, which reduces the cost of serving LLM-based agents by caching and reusing structured plan templates.

Sources

Collaborative LLM Inference via Planning for Efficient Reasoning

Towards a Cascaded LLM Framework for Cost-effective Human-AI Decision-Making

Understanding Input Selectivity in Mamba: Impact on Approximation Power, Memorization, and Associative Recall Capacity

'Memory States' from Almost Nothing: Representing and Computing in a Non-associative Algebra

Scaling Algorithm Distillation for Continuous Control with Mamba

AgentDistill: Training-Free Agent Distillation with Generalizable MCP Boxes

Cost-Efficient Serving of LLM Agents via Test-Time Plan Caching

Emergence of Primacy and Recency Effect in Mamba: A Mechanistic Point of View

Built with on top of