The field of stochastic optimization and deep learning is rapidly evolving, with a focus on developing adaptive algorithms that can handle complex problems with interdependent decision variables and objectives. Recent research has made significant progress in this area, with the development of novel algorithms that can achieve sharp convergence rates without prior knowledge of the noise level. These algorithms have the potential to improve the efficiency and effectiveness of stochastic optimization methods, particularly in high-noise regimes. Notably, the use of momentum normalization techniques and adaptive parameter choices has been shown to be effective in achieving optimal convergence rates. Additionally, there has been a growing interest in understanding the role of learning rate schedules in deep learning, with research highlighting the importance of functional scaling laws in characterizing the evolution of population risk during training. Furthermore, the development of unified noise-curvature views of loss of trainability has led to the creation of simple per-layer schedulers that can stabilize training and improve accuracy. Some noteworthy papers in this area include: The paper on Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization, which proposes novel adaptive algorithms for nonconvex-strongly-concave minimax optimization and nonconvex-strongly-convex bilevel optimization. The paper on Unveiling the Role of Learning Rate Schedules via Functional Scaling Laws, which introduces a functional scaling law that characterizes the evolution of population risk during training for general learning rate schedules.