The field of neural network optimization is moving towards more efficient and scalable methods. Recent developments have focused on improving the performance of deep neural networks while reducing their computational costs and environmental impact. One of the key directions is the development of modularization techniques that enable the reuse of pre-trained models, reducing the need for redundant computations. Another important area of research is the development of sparsification strategies that can effectively prune neural networks without compromising their accuracy. Additionally, there is a growing interest in optimizing neural networks for specific tasks, such as generative face video coding, and developing more efficient algorithms for training and deploying neural networks. Notable papers in this area include: NeMo, which proposes a neuron-level modularizing-while-training approach for decomposing DNN models, achieving average gains of 1.72% in module classification accuracy and 58.10% reduction in module size. ONG, which introduces a one-shot NMF-based gradient masking strategy for efficient model sparsification, demonstrating comparable or superior performance to established stable sparsification methods. A Convergent Primal-Dual Algorithm, which presents a novel theoretical framework for computing the information RDP function, establishing a rigorous convergence rate of O(1/n) for the computation of RDP functions. A Lightweight Dual-Mode Optimization, which proposes a dual-mode optimization framework for generative face video coding, achieving 90.4% parameter reduction and 88.9% computation saving compared to the baseline. GRAFT, which introduces a scalable in-training subset selection method, preserving the training trajectory while reducing wall-clock time, energy consumption, and CO2 emissions. One Shot vs. Iterative, which presents a systematic comparison of one-shot and iterative pruning strategies, finding that each method has specific advantages and introducing a hybrid approach that can outperform traditional methods. Formal Algorithms for Model Efficiency, which introduces the Knob-Meter-Rule framework, a unified formalism for representing and reasoning about model efficiency techniques in deep learning. Optimal Subspace Embeddings, which gives a proof of the conjecture of Nelson and Nguyen on the optimal dimension and sparsity of oblivious subspace embeddings, up to sub-polylogarithmic factors.