Abstract:Both neuroimaging and genomics datasets are often gathered for the detection of neurodegenerative diseases. Huge dimensionalities of neuroimaging data as well as omics data pose tremendous challenge for methods integrating multiple modalities. There are few existing solutions that can combine both multi-modal imaging and multi-omics datasets to derive neurological insights. We propose a deep neural network architecture that combines both structural and functional connectome data with multi-omics data for disease classification. A graph convolution layer is used to model functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) data simultaneously to learn compact representations of the connectome. A separate set of graph convolution layers are then used to model multi-omics datasets, expressed in the form of population graphs, and combine them with latent representations of the connectome. An attention mechanism is used to fuse these outputs and provide insights on which omics data contributed most to the model's classification decision. We demonstrate our methods for Parkinson's disease (PD) classification by using datasets from the Parkinson's Progression Markers Initiative (PPMI). PD has been shown to be associated with changes in the human connectome and it is also known to be influenced by genetic factors. We combine DTI and fMRI data with multi-omics data from RNA Expression, Single Nucleotide Polymorphism (SNP), DNA Methylation and non-coding RNA experiments. A Matthew Correlation Coefficient of greater than 0.8 over many combinations of multi-modal imaging data and multi-omics data was achieved with our proposed architecture. To address the paucity of paired multi-modal imaging data and the problem of imbalanced data in the PPMI dataset, we compared the use of oversampling against using CycleGAN on structural and functional connectomes to generate missing imaging modalities. Furthermore, we performed ablation studies that offer insights into the importance of each imaging and omics modality for the prediction of PD. Analysis of the generated attention matrices revealed that DNA Methylation and SNP data were the most important omics modalities out of all the omics datasets considered. Our work motivates further research into imaging genetics and the creation of more multi-modal imaging and multi-omics datasets to study PD and other complex neurodegenerative diseases.

Multimodal functional deep learning for multiomics data

Deep Multi-modal Latent Representation Learning for Automated Dementia Diagnosis

A deep generative multimodal imaging genomics framework for Alzheimer's disease prediction

Multi-scale multimodal deep learning framework for Alzheimer's disease diagnosis

Alzheimer’s Disease Diagnosis from Multi-Modal Data Via Feature Inductive Learning and Dual Multilevel Graph Neural Network

Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis

Patch-based Deep Multi-Modal Learning Framework for Alzheimer’s Disease Diagnosis Using Multi-View Neuroimaging

Supervised Multi-Modal Fission Learning

Feature Learning and Fusion of Multimodality Neuroimaging and Genetic Data for Multi-status Dementia Diagnosis

Deep Multi-Modal Discriminative and Interpretability Network for Alzheimer's Disease Diagnosis

Multimodal Brain Disease Classification with Functional Interaction Learning from Single fMRI Volume

Multimodal deep learning models for early detection of Alzheimer’s disease stage

Alzheimer's disease diagnosis from multi-modal data via feature inductive learning and dual multilevel graph neural network

Incomplete Multimodal Learning for Complex Brain Disorders Prediction

Deep Fusion of Brain Structure-Function in Mild Cognitive Impairment

Multi-Hypergraph Learning for Incomplete Multimodality Data

Hypergraph-regularized multimodal learning by graph diffusion for imaging genetics based Alzheimer's Disease diagnosis

MDMA: Multimodal Data and Multi-attention Based Deep Learning Model for Alzheimer’s Disease Diagnosis

Combining Neuroimaging and Omics Datasets for Disease Classification Using Graph Neural Networks

Effective Feature Learning and Fusion of Multimodality Data Using Stage-Wise Deep Neural Network for Dementia Diagnosis

Deep Multimodal Predictome for Studying Mental Disorders