The field of large language models (LLMs) is moving towards more efficient and cost-effective deployment, with a focus on dynamic selection, test-time scaling, and pruning techniques. Researchers are exploring innovative methods to balance performance and cost, such as cross-attention routing, latency-aware test-time compute, and reasoning-aware compression. These advances have the potential to significantly improve the scalability and usability of LLMs in real-world applications. Noteworthy papers include: One Head, Many Models, which introduces a unified routing framework for dynamic LLM selection, and A1: Asynchronous Test-Time Scaling, which proposes a statistically guaranteed adaptive inference framework for scalable LLM inference. Additionally, papers like Reasoning Models Can be Accurately Pruned Via Chain-of-Thought Reconstruction and EconProver: Towards More Economical Test-Time Scaling for Automated Theorem Proving demonstrate significant improvements in efficiency and performance.
Efficient Large Language Model Deployment and Reasoning
Sources
One Head, Many Models: Cross-Attention Routing for Cost-Aware LLM Selection
Latency and Token-Aware Test-Time Compute
Reasoning Models Can be Accurately Pruned Via Chain-of-Thought Reconstruction
EconProver: Towards More Economical Test-Time Scaling for Automated Theorem Proving
Large Language Models Imitate Logical Reasoning, but at what Cost?
{\L}ukasiewicz Logic with Actions for Neural Networks training
Understanding the Thinking Process of Reasoning Models: A Perspective from Schoenfeld's Episode Theory
Theorem Provers: One Size Fits All?
A1: Asynchronous Test-Time Scaling via Conformal Prediction