The field of large language models is moving towards more efficient and effective training methods, with a focus on low-precision training and optimization. Recent research has made significant progress in understanding the theoretical foundations of low-precision training, including the development of new frameworks for analyzing the convergence of adaptive optimizers under floating-point quantization. Additionally, there is a growing interest in exploring alternative quantization formats, such as integer quantization, which has shown promise in achieving better accuracy and efficiency in certain scenarios. Another area of research is the development of new optimization techniques, such as backward-friendly optimization, which enables efficient training of large language models with approximate gradients under memory constraints. Noteworthy papers in this area include A Convergence Analysis of Adaptive Optimizers under Floating-point Quantization, which derives convergence rates for adaptive optimizers under floating-point quantization, and INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formats, which systematically investigates the trade-offs between floating-point and integer quantization formats. Furthermore, researchers are also exploring new methods for explaining and mitigating instability in transformers, such as Numerical Fragility in Transformers: A Layer-wise Theory for Explaining, Forecasting, and Mitigating Instability, which provides a unified forward-stability bound for transformers. Overall, these advances have the potential to significantly improve the efficiency and effectiveness of large language models, and are likely to have a major impact on the field in the coming years.