Multimodal Human Behavior Analysis

The field of human behavior analysis is moving towards a more comprehensive understanding of emotional and physiological signals through multimodal approaches. Recent studies have highlighted the importance of combining visual, audio, and physiological modalities to enhance the accuracy and robustness of emotion recognition, intent understanding, and health monitoring. Noteworthy papers include MPFNet, which achieves state-of-the-art performance in micro-expression recognition by leveraging a progressive training strategy and multi-prior fusion. The MMME dataset is also notable, providing a comprehensive collection of multimodal micro-expression data that enables synchronized analysis of facial, central nervous system, and peripheral physiological signals. These innovative approaches and datasets are driving the field towards a more nuanced understanding of human behavior and have significant implications for applications in healthcare, human-computer interaction, and affective computing.

Sources

Non-Contact Health Monitoring During Daily Personal Care Routines

MPFNet: A Multi-Prior Fusion Network with a Progressive Training Strategy for Micro-Expression Recognition

MMME: A Spontaneous Multi-Modal Micro-Expression Dataset Enabling Visual-Physiological Fusion

Multimodal Emotion Coupling via Speech-to-Facial and Bodily Gestures in Dyadic Interaction

WDMIR: Wavelet-Driven Multimodal Intent Recognition

Built with on top of