Abstract:Multimodal data analysis has attracted ever-increasing attention in computational biology and bioinformatics community recently. However, existing multimodal learning approaches need all data modalities available at both training and prediction stages, thus they cannot be applied to many real-world biomedical applications, which often have a missing modality problem as the collection of all modalities is prohibitively costly. Meanwhile, two diagnosis-related pieces of information are of main interest during the examination of a subject regarding a chronic disease (with longitudinal progression): their current status (diagnosis) and how it will change before next visit (longitudinal outcome). Correct responses to these queries can identify susceptible individuals and provide the means of early interventions for them. In this article, we develop a novel adversarial mutual learning framework for longitudinal disease progression prediction, allowing us to leverage multiple data modalities available for training to train a performant model that uses a single modality for prediction. Specifically, in our framework, a single-modal model (which utilizes the main modality) learns from a pretrained multimodal model (which accepts both main and auxiliary modalities as input) in a mutual learning manner to (1) infer outcome-related representations of the auxiliary modalities based on its own representations for the main modality during adversarial training and (2) successfully combine them to predict the longitudinal outcome. We apply our method to analyze the retinal imaging genetics for the early diagnosis of age-related macular degeneration (AMD) disease, that is, simultaneous assessment of the severity of AMD at the time of the current visit and the prognosis of the condition at the subsequent visit. Our experiments using the Age-Related Eye Disease Study dataset show that our method is more effective than baselines at classifying patients' current and forecasting their future AMD severity.

Incomplete Multimodal Learning for Complex Brain Disorders Prediction

Feature-Based Transformer with Incomplete Multimodal Brain Images for Diagnosis of Neurodegenerative Diseases

Multimodal transformer network for incomplete image generation and diagnosis of Alzheimer's disease

Incomplete Learning of Multi-Modal Connectome for Brain Disorder Diagnosis Via Modal-Mixup and Deep Supervision.

Inter-modality Dependence Induced Data Recovery for MCI Conversion Prediction

Multi-Hypergraph Learning for Incomplete Multimodality Data

A deep generative multimodal imaging genomics framework for Alzheimer's disease prediction

Multi-source Feature Learning for Joint Analysis of Incomplete Multiple Heterogeneous Neuroimaging Data

Deep Multimodal Predictome for Studying Mental Disorders

Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes

Integration of Multimodal Data for Deciphering Brain Disorders.

Multimodal Predictive Modeling: Scalable Imaging Informed Approaches to Predict Future Brain Health

Cascaded Multi-Modal Mixing Transformers for Alzheimer's Disease Classification with Incomplete Data

Latent Representation Learning for Alzheimer's Disease Diagnosis with Incomplete Multi-Modality Neuroimaging and Genetic Data.

Incomplete multi-modal representation learning for Alzheimer's disease diagnosis

A Holistic Approach to Multimodal Brain Disease Diagnosis in Imbalanced Data

Multimodal deep learning models for early detection of Alzheimer’s disease stage

A Novel Transfer Learning Model for Predictive Analytics Using Incomplete Multimodality Data.

Multimodal Genotype and Phenotype Data Integration to Improve Partial Data-Based Longitudinal Prediction

A Cross-Modal Mutual Knowledge Distillation Framework for Alzheimer's Disease Diagnosis: Addressing Incomplete Modalities

Feature Learning and Fusion of Multimodality Neuroimaging and Genetic Data for Multi-status Dementia Diagnosis