The field of vision-language models is rapidly advancing, with a focus on developing more robust and generalizable models for real-world scene understanding. Recent works have explored the use of multimodal foundation models, vision-language integration, and dynamic context-aware scene reasoning to improve zero-shot learning and adaptation to new environments. These approaches have shown significant gains in object recognition, activity detection, and scene captioning, and have the potential to enable more effective and efficient scene understanding in a variety of applications. Notable papers in this area include TokenCLIP, which proposes a token-wise adaptation framework for fine-grained anomaly learning, and Representation-Level Counterfactual Calibration, which introduces a counterfactual approach to debias zero-shot recognition. Overall, the field is moving towards more advanced and specialized models that can handle the complexity and variability of real-world scenes.
Advances in Vision-Language Models for Real-World Scene Understanding
Sources
ZING-3D: Zero-shot Incremental 3D Scene Graphs via Vision-Language Models
TokenCLIP: Token-wise Prompt Learning for Zero-shot Anomaly Detection
Bridging the gap to real-world language-grounded visual concept learning
Semantic Relation-Enhanced CLIP Adapter for Domain Adaptive Zero-Shot Learning
LOC: A General Language-Guided Framework for Open-Set 3D Occupancy Prediction
Cross-Species Transfer Learning in Agricultural AI: Evaluating ZebraPose Adaptation for Dairy Cattle Pose Estimation
Seeing the Unseen: Towards Zero-Shot Inspection for Wind Turbine Blades using Knowledge-Augmented Vision Language Models
Gen-LangSplat: Generalized Language Gaussian Splatting with Pre-Trained Feature Compression
Finding 3D Scene Analogies with Multimodal Foundation Models
Vision-Language Integration for Zero-Shot Scene Understanding in Real-World Environments
Visual Diversity and Region-aware Prompt Learning for Zero-shot HOI Detection
Towards Fine-Grained Vision-Language Alignment for Few-Shot Anomaly Detection
Representation-Level Counterfactual Calibration for Debiased Zero-Shot Recognition
Dynamic Context-Aware Scene Reasoning Using Vision-Language Alignment in Zero-Shot Real-World Scenarios