Multimodal Analysis in Medical Research

The field of medical research is witnessing a significant shift towards multimodal analysis, where diverse types of data are integrated to improve prognosis, prediction, and treatment outcomes. This approach has been particularly effective in cancer research, where the combination of genomic, pathological, and clinical data has led to more accurate biomarker prediction and survival analysis. The use of graph neural networks and heterogeneous graph neural networks has also shown promise in modeling complex interactions between different data modalities. Furthermore, the development of novel frameworks and models, such as online distillation and decoupling-reorganization-fusion, has enabled more effective fusion of multimodal data and improved the generalization ability of predictive models. Noteworthy papers in this area include:

  • A paper proposing an online distillation approach based on Multi-modal Knowledge Decomposition for biomarker prediction in breast cancer histopathology, which achieves superior performance using uni-modal data.
  • A paper presenting a graph neural network model with mutual information and global fusion for multimodal medical prognosis, which surpasses existing methods on datasets related to liver prognosis and the METABRIC study.
  • A paper proposing a novel Decoupling-Reorganization-Fusion framework for cancer survival prediction, which increases the diversity of feature combinations and granularity, enhancing the generalization ability of the subsequent expert networks.
  • A paper proposing a heterogeneous graph neural network model to predict potential drug-drug interactions, which outperforms state-of-the-art baselines in prediction accuracy and robustness.

Sources

Multi-modal Knowledge Decomposition based Online Distillation for Biomarker Prediction in Breast Cancer Histopathology

GraphMMP: A Graph Neural Network Model with Mutual Information and Global Fusion for Multimodal Medical Prognosis

Decouple, Reorganize, and Fuse: A Multimodal Framework for Cancer Survival Prediction

Predicting Drug-Drug Interactions Using Heterogeneous Graph Neural Networks: HGNN-DDI

Built with on top of