The field of large language models (LLMs) is moving towards more efficient inference methods, with a focus on adaptive and controllable test-time compute. Recent work has introduced novel training paradigms, such as dynamic memorization and exploration, and probabilistic frameworks for inference-time scaling. These approaches aim to reduce the computational resources required for LLMs while maintaining or improving their performance. Notably, some studies have explored the use of energy-based transformers, which can learn to think solely from unsupervised learning, and have shown promising results in terms of scalability and performance. Additionally, auto-route switching frameworks have been proposed to dynamically assign input queries to either thinking or non-thinking modes, optimizing accuracy, cost-efficiency, and user experience. Overall, the field is shifting towards more efficient and adaptive inference methods, which is expected to have a significant impact on the deployment of LLMs in real-world applications. Noteworthy papers include: Probabilistic Optimality for Inference-time Scaling, which derives a theoretical lower bound on the required number of samples to achieve a target performance level. Energy-Based Transformers are Scalable Learners and Thinkers, which introduces a new class of energy-based models that can learn to think solely from unsupervised learning. SynapseRoute: An Auto-Route Switching Framework on Dual-State Large Language Model, which proposes a machine learning-based dynamic routing framework that intelligently assigns input queries to either thinking or non-thinking modes.