The field of deep learning is witnessing significant advancements in optimization techniques and language modeling. Researchers are exploring innovative methods to accelerate deep neural network training, reduce memory burden, and improve compute efficiency. One notable direction is the development of hybrid order optimization techniques that leverage both gradient and curvature information to enhance convergence speeds. Additionally, there is a growing interest in optimizing language models for specific languages and improving their parameter efficiency. These advancements have the potential to unlock more efficient training procedures, enable the deployment of models on resource-constrained devices, and promote the development of high-quality language AI for less-represented languages. Noteworthy papers include: The paper on Distributed Hybrid Order Optimization, which presents a novel distributed design for accelerating DNN training with a low memory burden. The CompleteP paper, which introduces a parameterization that achieves depth-wise hyperparameter transfer and non-lazy learning. The Bielik 11B v2 and Bielik v3 papers, which demonstrate exceptional performance and parameter efficiency in Polish language processing.