The field of large language models (LLMs) is rapidly advancing, with a focus on improving reasoning capabilities and reducing hallucinations. Recent developments have introduced new paradigms, such as cognitive loops and logic-augmented generation, which enable LLMs to self-formulate ways of approaching problems and provide more accurate and transparent results. Additionally, techniques like certainty-guided reflection suppression and deliberative reasoning networks have been proposed to mitigate overthinking and improve reasoning fidelity. These innovations have significant implications for AI safety and scientific discovery, and are expected to continue shaping the direction of LLM research. Noteworthy papers include: Cognitive Loop via In-Situ Optimization, which enables LLMs to self-formulate ways of approaching problems and provides a final belief or answer. Deliberative Reasoning Network, which reframes logical reasoning from probability maximization to uncertainty minimization and achieves intrinsic interpretability by explicitly tracking belief states and quantifying epistemic uncertainty.
Advances in Large Language Models
Sources
Out-of-Context Abduction: LLMs Make Inferences About Procedural Data Leveraging Declarative Facts in Earlier Training Data
Cognitive Loop via In-Situ Optimization: Self-Adaptive Reasoning for Science
Enhancing Japanese Large Language Models with Reasoning Vectors
RegMean++: Enhancing Effectiveness and Generalization of Regression Mean for Model Merging
Long Story Generation via Knowledge Graph and Literary Theory
RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Prior
Thinking with Nothinking Calibration: A New In-Context Learning Paradigm in Reasoning Large Language Models
MultiRAG: A Knowledge-guided Framework for Mitigating Hallucination in Multi-source Retrieval Augmented Generation
Markov Chain Estimation with In-Context Learning
Method-Based Reasoning for Large Language Models: Extraction, Reuse, and Continuous Improvement
Deliberative Reasoning Network: An Uncertainty-Driven Paradigm for Belief-Tracked Inference with Pretrained Language Models
Hop, Skip, and Overthink: Diagnosing Why Reasoning Models Fumble during Multi-Hop Analysis
Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression
LAG: Logic-Augmented Generation from a Cartesian Perspective