The field of cross-modal learning and knowledge distillation is witnessing significant developments, with a focus on improving the efficiency and effectiveness of models in transferring knowledge across different modalities. Researchers are exploring innovative approaches to address the challenges posed by the modality gap, such as the use of bidirectional knowledge distillation mechanisms and data-dependent regularizers. These advancements have the potential to enhance the performance of models in various applications, including facial landmark detection and multimodal learning. Noteworthy papers in this area include:

• A study on optimal regularization for performative learning, which shows that regularization can help cope with performative effects in high-dimensional ridge regression.
• A work on information-theoretic criteria for knowledge distillation in multimodal learning, which introduces the Cross-modal Complementarity Hypothesis to inform the selection of optimal teacher modalities.
• A paper on rethinking knowledge distillation, which highlights the importance of understanding knowledge transfer mechanisms and the potential risks of asymmetric knowledge transfer.