The field of neural networks is moving towards improving robustness and generalization, with a focus on developing innovative methods to enhance model reliability and accuracy. Recent studies have explored the benefits of sharpness-aware minimization, neuro-inspired front-ends, and distributional input projection networks in improving model calibration and robustness. Noteworthy papers include 'Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization', which proposes a variant of sharpness-aware minimization to ameliorate model calibration, and 'Explicitly Modeling Subcortical Vision with a Neuro-Inspired Front-End Improves CNN Robustness', which introduces a novel front-end block that mimics the primate primary visual cortex to improve model robustness. Another notable paper is 'Towards Better Generalization via Distributional Input Projection Network', which presents a novel framework that projects inputs into learnable distributions at each layer to induce a smoother loss landscape and promote better generalization.
Advances in Neural Network Robustness and Generalization
Sources
Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization
A comparative analysis of a neural network with calculated weights and a neural network with random generation of weights based on the training dataset size
The Butterfly Effect in Pathology: Exploring Security in Pathology Foundation Models
LightSAM: Parameter-Agnostic Sharpness-Aware Minimization
On the Lipschitz Continuity of Set Aggregation Functions and Neural Networks for Sets
A Flat Minima Perspective on Understanding Augmentations and Model Robustness
Improving Knowledge Distillation Under Unknown Covariate Shift Through Confidence-Guided Data Augmentation
AERO: A Redirection-Based Optimization Framework Inspired by Judo for Robust Probabilistic Forecasting
On the Robustness of Tabular Foundation Models: Test-Time Attacks and In-Context Defenses
Explicitly Modeling Subcortical Vision with a Neuro-Inspired Front-End Improves CNN Robustness
Impact of Tuning Parameters in Deep Convolutional Neural Network Using a Crack Image Dataset
Do Neural Networks Need Gradient Descent to Generalize? A Theoretical Study
Average Calibration Losses for Reliable Uncertainty in Medical Image Segmentation
Half-Layered Neural Networks
KOALA++: Efficient Kalman-Based Optimization of Neural Networks with Gradient-Covariance Products
Grokking and Generalization Collapse: Insights from \texttt{HTSR} theory
Orthogonal Gradient Descent Improves Neural Calibration
Towards Better Generalization via Distributional Input Projection Network
Robustness as Architecture: Designing IQA Models to Withstand Adversarial Perturbations