The field of machine learning is moving towards more efficient and scalable architectures, with a focus on accelerating model training and improving energy efficiency. Recent developments have explored novel adaptive batch size algorithms, neural network expansion methods, and hardware-software co-designs to achieve state-of-the-art performance. Notably, innovative approaches have been proposed to prevent neuron inactivity during network expansion and to leverage physical hardware properties to improve energy efficiency. Furthermore, the integration of optical and photonic technologies has enabled the development of high-speed and low-latency computing systems.

Noteworthy papers include: DIVEBATCH, which proposes a gradient-diversity aware batch size adaptation algorithm to accelerate model training. Otters, which introduces an energy-efficient spiking transformer via optical time-to-first-spike encoding, achieving state-of-the-art accuracy and improved energy efficiency. High Clockrate Free-space Optical In-Memory Computing, which presents a fanout spatial time-of-flight optical neural network for fast and low-latency computing.