The field of physics simulations is rapidly advancing with the development of innovative transformer architectures and neural emulators. Recent research has focused on creating more scalable and versatile transformer models that can efficiently handle large-scale simulations, such as those involving partial differential equations (PDEs). These new architectures have shown improved performance over state-of-the-art models and can be used as the backbone for building large-scale foundation models in physical sciences. Additionally, neural emulators are being developed to enhance long-term prediction accuracy and stability in complex dynamical systems, such as turbulent fluid dynamics. These models leverage hierarchical representations and implicit neural networks to capture dynamics across multiple granularities and enforce long-range temporal coherence. Another area of advancement is in time series prediction, where neural networks are being combined with stochastic differential equations to model non-stationary data and abrupt changes. Furthermore, new solvers are being introduced to efficiently solve high-dimensional transport problems, which are fundamental in various scientific disciplines. These advancements have the potential to open new avenues for efficiently and accurately modeling complex phenomena. Noteworthy papers include: PDE-Transformer, which proposes an improved transformer-based architecture for surrogate modeling of physics simulations, and Hierarchical Implicit Neural Emulators, which introduces a multiscale implicit neural emulator for long-term prediction accuracy. Neural MJD and A Fast, Accurate and Oscillation-free Spectral Collocation Solver also demonstrate significant advancements in time series prediction and high-dimensional transport problems, respectively.