The field of large language models (LLMs) is moving towards developing more efficient and adaptive reasoning strategies. Recent research has focused on improving the computational efficiency of LLMs, particularly in real-time applications, by dynamically adjusting their reasoning depth and budget. This has led to the development of novel frameworks and methods that enable LLMs to optimize their reasoning processes, balancing efficiency and accuracy without compromising performance. Noteworthy papers in this regard include Aware First, Think Less, which introduces the Dynamic Reasoning-Boundary Self-Awareness Framework (DR. SAF) to improve the efficiency of LLMs, and Think in Blocks, which proposes an adaptive reasoning framework that enables LLMs to dynamically adjust the length of their reasoning processes based on task complexity. Additionally, papers such as Exploring Efficiency Frontiers of Thinking Budget in Medical Reasoning and OptimalThinkingBench have made significant contributions to the field by evaluating the efficiency of LLMs in medical reasoning tasks and introducing a unified benchmark to evaluate overthinking and underthinking in LLMs, respectively.
Efficient Reasoning in Large Language Models
Sources
Aware First, Think Less: Dynamic Boundary Self-Awareness Drives Extreme Reasoning Efficiency in Large Language Models
Exploring Efficiency Frontiers of Thinking Budget in Medical Reasoning: Scaling Laws between Computational Resources and Reasoning Quality
Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
Leveraging Large Language Models for Predictive Analysis of Human Misery
A Stitch in Time Saves Nine: Proactive Self-Refinement for Language Models
OptimalThinkingBench: Evaluating Over and Underthinking in LLMs
Input Time Scaling
Credence Calibration Game? Calibrating Large Language Models through Structured Play
Don't Think Twice! Over-Reasoning Impairs Confidence Calibration
Deep Think with Confidence
Think in Blocks: Adaptive Reasoning from Direct Response to Deep Reasoning