Improving Chain-of-Thought Reasoning in Large Language Models

The field of chain-of-thought reasoning in large language models is moving towards improving the reliability and efficiency of reasoning processes. Researchers are exploring methods to reduce redundancy in reasoning chains, calibrate the accuracy of intermediate steps, and fine-tune critical representations to enhance reasoning performance. Noteworthy papers in this area include:

  • Think Clearly: Improving Reasoning via Redundant Token Pruning, which demonstrates that removing redundant tokens in the reasoning process significantly improves performance.
  • Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning, which introduces a novel approach to calibrate CoT reasoning accuracy by leveraging the model's intrinsic veracity encoding.
  • Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning, which proposes a method to identify and optimize critical representations through information flow analysis.
  • Probabilistic Soundness Guarantees in LLM Reasoning Chains, which introduces a probabilistic framework to prevent error propagation in reasoning chains.

Sources

Think Clearly: Improving Reasoning via Redundant Token Pruning

Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey

Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning

Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning

Probabilistic Soundness Guarantees in LLM Reasoning Chains

Built with on top of