The field of neural architecture search is moving towards more efficient and robust methods, with a focus on simultaneously optimizing both architecture and weights of neural networks. Researchers are exploring new approaches, such as evolutionary algorithms and latent space optimization, to automate the design of high-performing neural networks. These methods have shown promising results in reducing computational costs and improving model performance. Notably, the use of sparse evolutionary training and motif-based structural optimization has been found to enhance the overall performance of sparse neural networks. Additionally, novel architectural variants, such as auto-compressing networks, have demonstrated improved noise robustness and better generalization capabilities. Overall, the field is advancing towards more efficient and effective neural architecture search methods.

Noteworthy papers in this area include:

• SWAT-NN, which proposes a framework for simultaneous weights and architecture training in a latent space.
• EMNAS-RL, which introduces evolutionary multi-objective network architecture search for reinforcement learning in autonomous driving.
• Auto-Compressing Networks, which showcases a unique property of auto-compression, enabling networks to organically compress information during training.