Abstract:In recent years, deep learning approaches have gained significant attention in predicting brain disorders using neuroimaging data. However, conventional methods often rely on single-modality data and supervised models, which provide only a limited perspective of the intricacies of the highly complex brain. Moreover, the scarcity of accurate diagnostic labels in clinical settings hinders the applicability of the supervised models. To address these limitations, we propose a novel self-supervised framework for extracting multiple representations from multimodal neuroimaging data to enhance group inferences and enable analysis without resorting to labeled data during pre-training. Our approach leverages Deep InfoMax (DIM), a self-supervised methodology renowned for its efficacy in learning representations by estimating mutual information without the need for explicit labels. While DIM has shown promise in predicting brain disorders from single-modality MRI data, its potential for multimodal data remains untapped. This work extends DIM to multimodal neuroimaging data, allowing us to identify disorder-relevant brain regions and explore multimodal links. We present compelling evidence of the efficacy of our multimodal DIM analysis in uncovering disorder-relevant brain regions, including the hippocampus, caudate, insula, - and multimodal links with the thalamus, precuneus, and subthalamus hypothalamus. Our self-supervised representations demonstrate promising capabilities in predicting the presence of brain disorders across a spectrum of Alzheimer's phenotypes. Comparative evaluations against state-of-the-art unsupervised methods based on autoencoders, canonical correlation analysis, and supervised models highlight the superiority of our proposed method in achieving improved classification performance, capturing joint information, and interpretability capabilities. The computational efficiency of the decoder-free strategy enhances its practical utility, as it saves compute resources without compromising performance. This work offers a significant step forward in addressing the challenge of understanding multimodal links in complex brain disorders, with potential applications in neuroimaging research and clinical diagnosis.

Using deep generative models for simultaneous representational and predictive modeling of brain and behavior: A graded supervised-to-unsupervised modeling framework.

Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes

Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Data

Multimodal Predictive Modeling: Scalable Imaging Informed Approaches to Predict Future Brain Health

Classifying Unconscious, Psychedelic, and Neuropsychiatric Brain States with Functional Connectivity, Graph Theory, and Cortical Gradient Analysis

Self-supervised multimodal learning for group inferences from MRI data: Discovering disorder-relevant brain regions and multimodal links

Generative Dynamical Models for Classification of rsfMRI Data

Surreal-GAN:Semi-Supervised Representation Learning via GAN for uncovering heterogeneous disease-related imaging patterns

Incomplete Multimodal Learning for Complex Brain Disorders Prediction

Deep latent variable joint cognitive modeling of neural signals and human behavior

A generative-discriminative framework that integrates imaging, genetic, and diagnosis into coupled low dimensional space

The roles of supervised machine learning in systems neuroscience

On self-supervised multi-modal representation learning: An application to Alzheimer's disease

Incorporating structured assumptions with probabilistic graphical models in fMRI data analysis

Disentangling brain heterogeneity via semi-supervised deep-learning and MRI: dimensional representations of Alzheimer's Disease

Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks

Generative discriminative models for multivariate inference and statistical mapping in medical imaging

Bayesian Models of Functional Connectomics and Behavior

Deep sr-DDL: Deep structurally regularized dynamic dictionary learning to integrate multimodal and dynamic functional connectomics data for multidimensional clinical characterizations

Probabilistic prediction of neurological disorders with a statistical assessment of neuroimaging data modalities

A Two-Part Mixed-Effects Modeling Framework For Analyzing Whole-Brain Network Data