Abstract:In recent years, deep learning models have been applied to neuroimaging data for early diagnosis of Alzheimer's disease (AD). Structural magnetic resonance imaging (sMRI) and positron emission tomography (PET) images provide structural and functional information about the brain, respectively. Combining these features leads to improved performance than using a single modality alone in building predictive models for AD diagnosis. However, current multi-modal approaches in deep learning, based on sMRI and PET, are mostly limited to convolutional neural networks, which do not facilitate integration of both image and phenotypic information of subjects. We propose to use graph neural networks (GNN) that are designed to deal with problems in non-Euclidean domains. In this study, we demonstrate how brain networks can be created from sMRI or PET images and be used in a population graph framework that can combine phenotypic information with imaging features of these brain networks. Then, we present a multi-modal GNN framework where each modality has its own branch of GNN and a technique is proposed to combine the multi-modal data at both the level of node vectors and adjacency matrices. Finally, we perform late fusion to combine the preliminary decisions made in each branch and produce a final prediction. As multi-modality data becomes available, multi-source and multi-modal is the trend of AD diagnosis. We conducted explorative experiments based on multi-modal imaging data combined with non-imaging phenotypic information for AD diagnosis and analyzed the impact of phenotypic information on diagnostic performance. Results from experiments demonstrated that our proposed multi-modal approach improves performance for AD diagnosis, and this study also provides technical reference and support the need for multivariate multi-modal diagnosis methods.

Hypergraph-regularized multimodal learning by graph diffusion for imaging genetics based Alzheimer's Disease diagnosis

Multi-Hypergraph Learning for Incomplete Multimodality Data

Multi-Modal Hypergraph Diffusion Network with Dual Prior for Alzheimer Classification

Feature Learning and Fusion of Multimodality Neuroimaging and Genetic Data for Multi-status Dementia Diagnosis

IG-GRD: A Model Based on Disentangled Graph Representation Learning for Imaging Genetic Data Fusion

Multi-Modal Data Analysis for Alzheimer’s Disease Diagnosis: An Ensemble Model Using Imagery and Genetic Features

Alzheimer’s Disease Diagnosis from Multi-Modal Data Via Feature Inductive Learning and Dual Multilevel Graph Neural Network

Multi-modal Graph Neural Network for Early Diagnosis of Alzheimer's Disease from sMRI and PET Scans

Alzheimer's disease diagnosis from multi-modal data via feature inductive learning and dual multilevel graph neural network

Deep joint learning diagnosis of Alzheimer's disease based on multimodal feature fusion

MMGK: Multimodality Multiview Graph Representations and Knowledge Embedding for Mild Cognitive Impairment Diagnosis

Identifying disease sensitive and quantitative trait-relevant biomarkers from multidimensional heterogeneous imaging genetics data via sparse multimodal multitask learning.

Multi-modal imaging genetics data fusion by deep auto-encoder and self-representation network for Alzheimer's disease diagnosis and biomarkers extraction

A deep generative multimodal imaging genomics framework for Alzheimer's disease prediction

Multimodal Hyperbolic Graph Learning for Alzheimer's Disease Detection

Exploration of Imaging Genetic Biomarkers of Alzheimer's Disease Based on a Machine Learning Method

Multi-Modal Diagnosis of Alzheimer's Disease using Interpretable Graph Convolutional Networks

Deep Multi-Modal Discriminative and Interpretability Network for Alzheimer's Disease Diagnosis

Multi-modal imaging genetics data fusion via a hypergraph-based manifold regularization: application to schizophrenia study

Multimodal data fusion based on IGERNNC algorithm for detecting pathogenic brain regions and genes in Alzheimer's disease

Probability Connectivity-Based Multimodality Regression Analysis for Associating Disease-Specific Multimodal Brain Imaging Phenotypes with Genetic Risk Factors.