Abstract:Both neuroimaging and genomics datasets are often gathered for the detection of neurodegenerative diseases. Huge dimensionalities of neuroimaging data as well as omics data pose tremendous challenge for methods integrating multiple modalities. There are few existing solutions that can combine both multi-modal imaging and multi-omics datasets to derive neurological insights. We propose a deep neural network architecture that combines both structural and functional connectome data with multi-omics data for disease classification. A graph convolution layer is used to model functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) data simultaneously to learn compact representations of the connectome. A separate set of graph convolution layers are then used to model multi-omics datasets, expressed in the form of population graphs, and combine them with latent representations of the connectome. An attention mechanism is used to fuse these outputs and provide insights on which omics data contributed most to the model's classification decision. We demonstrate our methods for Parkinson's disease (PD) classification by using datasets from the Parkinson's Progression Markers Initiative (PPMI). PD has been shown to be associated with changes in the human connectome and it is also known to be influenced by genetic factors. We combine DTI and fMRI data with multi-omics data from RNA Expression, Single Nucleotide Polymorphism (SNP), DNA Methylation and non-coding RNA experiments. A Matthew Correlation Coefficient of greater than 0.8 over many combinations of multi-modal imaging data and multi-omics data was achieved with our proposed architecture. To address the paucity of paired multi-modal imaging data and the problem of imbalanced data in the PPMI dataset, we compared the use of oversampling against using CycleGAN on structural and functional connectomes to generate missing imaging modalities. Furthermore, we performed ablation studies that offer insights into the importance of each imaging and omics modality for the prediction of PD. Analysis of the generated attention matrices revealed that DNA Methylation and SNP data were the most important omics modalities out of all the omics datasets considered. Our work motivates further research into imaging genetics and the creation of more multi-modal imaging and multi-omics datasets to study PD and other complex neurodegenerative diseases.

A Multimodal Foundation Model for Discovering Genetic Associations with Brain Imaging Phenotypes

Incomplete Multimodal Learning for Complex Brain Disorders Prediction

Associating Multi-Modal Brain Imaging Phenotypes and Genetic Risk Factors Via a Dirty Multi-Task Learning Method.

Multimodal and multicontrast image fusion via deep generative models

Interpretable Multimodal Fusion Networks Reveal Mechanisms of Brain Cognition

Integrating Multimodal Neuroimaging and Genetics: A Structurally-Linked Sparse Canonical Correlation Analysis Approach

DeepGAMI: deep biologically guided auxiliary learning for multimodal integration and imputation to improve genotype–phenotype prediction

A Dirty Multi-task Learning Method for Multi-modal Brain Imaging Genetics

Self-supervised multimodal neuroimaging yields predictive representations for a spectrum of Alzheimer's phenotypes

Machine Learning for Brain Imaging Genomics Methods: A Review

Identifying Diagnosis-Specific Genotype-Phenotype Associations Via Joint Multitask Sparse Canonical Correlation Analysis and Classification

Mapping Multi-Modal Brain Connectome for Brain Disorder Diagnosis Via Cross-Modal Mutual Learning.

Identifying Associations among Genomic, Proteomic and Imaging Biomarkers Via Adaptive Sparse Multi-View Canonical Correlation Analysis

Multi-modal Imaging Genomics Transformer: Attentive Integration of Imaging with Genomic Biomarkers for Schizophrenia Classification

Multi-modal deep learning of functional and structural neuroimaging and genomic data to predict mental illness

Multimodal Predictive Modeling: Scalable Imaging Informed Approaches to Predict Future Brain Health

Self-supervised multimodal learning for group inferences from MRI data: Discovering disorder-relevant brain regions and multimodal links

Identifying Multimodal Intermediate Phenotypes Between Genetic Risk Factors and Disease Status in Alzheimer's Disease

Combining Neuroimaging and Omics Datasets for Disease Classification Using Graph Neural Networks

Multimodal foundation models are better simulators of the human brain

An interpretable generative multimodal neuroimaging-genomics framework for decoding Alzheimer's disease