The field of vision-language models is moving towards a deeper understanding of the underlying structures and relationships between visual and linguistic representations. Recent research has focused on exploring the compositionality of visual embeddings, spatial awareness, and geometry-aware architectures. These innovations have the potential to enhance the interpretability and performance of vision-language models in various tasks, such as compositional classification, group robustness, and spatial reasoning. Noteworthy papers in this area include: Galaxy Walker, which introduces a geometry-aware vision-language model for galaxy-scale understanding tasks, achieving state-of-the-art performance in galaxy property estimation and morphology classification tasks. Beyond Semantics also presents a significant contribution by identifying the limitations of current vision-language models in spatial reasoning tasks and proposing interpretable interventions to restore spatial awareness.