The field of large language models (LLMs) is moving towards improving chain-of-thought (CoT) reasoning, with a focus on increasing efficiency, accuracy, and adaptability. Researchers are exploring various techniques to mitigate overthinking, improve concise reasoning, and enhance the planning capabilities of LLMs. Noteworthy papers include: Stop When Enough, which presents a training-free framework to adaptively determine when to stop reasoning, reducing token usage by 20-55% while maintaining or improving accuracy. Concise Reasoning in the Lens of Lagrangian Optimization, which introduces a principled and pragmatic strategy to generate only essential intermediate steps, reducing output length by 65% and improving accuracy by 15%. Enhancing LLM Reasoning via Non-Human-Like Reasoning Path Preference Optimization, which proposes a method to leverage a confidence signal to identify points of maximal uncertainty and applies self-generated, non-human-like reasoning-path guidance to mitigate trajectory drift. Information-Preserving Reformulation of Reasoning Traces for Antidistillation, which presents a two-step reformulation to disrupt distillation across student models of different sizes and types on various reasoning benchmarks. Enhancing Long Chain-of-Thought Reasoning through Multi-Path Plan Aggregation, which proposes a framework to augment single-pass reasoning with plan exploration and aggregation, improving planning quality and achieving state-of-the-art results under a substantially lower training budget. DeepPlanner, which effectively enhances the planning capabilities of deep research agents through advantage shaping, improving planning quality and achieving state-of-the-art results under a substantially lower training budget. CoT-Evo, which proposes an evolutionary CoT distillation framework to construct a diverse pool of reasoning trajectories, enrich them with automatically retrieved domain knowledge, and iteratively refine the trajectories using novelty-driven selection, reflective recombination, and mutation. Attention Illuminates LLM Reasoning, which positions attention as a privileged substrate to render the internal logic of LLMs legible and introduces three novel RL strategies to dynamically perform targeted credit assignment to critical nodes. Less is More, which proposes Minimal Test-Time Intervention (MTI), a training-free framework that enhances reasoning accuracy and stability with minimal overhead.
Advances in Chain-of-Thought Reasoning for Large Language Models
Sources
Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoning
Concise Reasoning in the Lens of Lagrangian Optimization
Enhancing LLM Reasoning via Non-Human-Like Reasoning Path Preference Optimization
Information-Preserving Reformulation of Reasoning Traces for Antidistillation
Enhancing Long Chain-of-Thought Reasoning through Multi-Path Plan Aggregation
LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokens
Self-Verifying Reflection Helps Transformers with CoT Reasoning
Towards Inference-time Scaling for Continuous Space Reasoning
DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping
CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning
Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization
Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention