The field of multimodal learning and image quality assessment is witnessing significant developments, with a focus on addressing the challenges of modality gaps, dimension collapse, and contrast distortion. Researchers are exploring innovative approaches to analyze and improve the alignment of representations from different modalities, leading to enhanced performance in downstream tasks. Notably, the introduction of theoretical frameworks and novel loss functions is facilitating the development of more robust and generalizable models. Furthermore, the application of deep learning-based methods to image quality assessment is yielding promising results, with improvements in prediction accuracy and stability. The use of contrast enhancement algorithms and pseudo-reference images is also showing potential in addressing the issue of contrast distortion. Overall, these advancements are paving the way for more effective and efficient multimodal learning and image quality assessment systems. Noteworthy papers include: DECOR, which introduces a deep clustering framework with orientation robustness for wafer defect detection, and μDeepIQA, which proposes a deep learning-based image quality assessment method for optical microscopy. Additionally, the Chem-NMF method applies a physical chemistry perspective to improve the convergence of non-negative matrix factorization algorithms, while the SpherePair loss function enables angular constraint embedding for constrained clustering.
Advancements in Multimodal Learning and Image Quality Assessment
Sources
Decrypt Modality Gap in Multimodal Contrastive Learning: From Convergent Representation to Pair Alignment
\mu DeepIQA: deep learning-based fast and robust image quality assessment with local predictions for optical microscopy
No-reference Quality Assessment of Contrast-distorted Images using Contrast-enhanced Pseudo Reference