Abstract:Alzheimer's disease (AD) is a neurodegenerative disorder, the most common cause of dementia, so the accurate diagnosis of AD and its prodromal stage mild cognitive impairment (MCI) is significant. Recent studies have demonstrated that multiple neuroimaging and biological measures contain complementary information for diagnosis. Many existing multi-modal models based on deep learning simply concatenate each modality's features despite substantial differences in representation spaces. In this paper, we propose a novel multi-modal cross-attention AD diagnosis (MCAD) framework to learn the interaction between modalities for better playing their complementary roles for AD diagnosis with multi-modal data including structural magnetic resonance imaging (sMRI), fluorodeoxyglucose-positron emission tomography (FDG-PET) and cerebrospinal fluid (CSF) biomarkers. Specifically, the imaging and non-imaging representations are learned by the image encoder based on cascaded dilated convolutions and CSF encoder, respectively. Then, a multi-modal interaction module is introduced, which takes advantage of cross-modal attention to integrate imaging and non-imaging information and reinforce relationships between these modalities. Moreover, an extensive objective function is designed to reduce the discrepancy between modalities for effectively fusing the features of multi-modal data, which could further improve the diagnosis performance. We evaluate the effectiveness of our proposed method on the ADNI dataset, and the extensive experiments demonstrate that our MCAD achieves superior performance for multiple AD-related classification tasks, compared to several competing methods. Also, we investigate the importance of cross-attention and the contribution of each modality to the diagnostics performance. The experimental results demonstrate that combining multi-modality data via cross-attention is helpful for accurate AD diagnosis.

CubeSat Deployable Antenna Using Bistable Composite Tape-Springs

The effect of N-allylnormorphine upon the antidiuretic action of morphine.

Effect of ketoconazole and miconazole on skeletal muscle mitochondrial calcium transport system

Multimodal Representations Learning and Adversarial Hypergraph Fusion for Early Alzheimer's Disease Prediction

Joint Association Discovery and Diagnosis of Alzheimer's Disease by Supervised Heterogeneous Multiview Learning.

Sparse Bayesian Multiview Learning for Simultaneous Association Discovery and Diagnosis of Alzheimer's Disease.

CMC: A consensus multi-view clustering model for predicting Alzheimer's disease progression

WISE 2014 Challenge: Multi-label Classification of Print Media Articles to Topics

Joint Learning Framework of Cross-Modal Synthesis and Diagnosis for Alzheimer's Disease by Mining Underlying Shared Modality Information.

Multimodal deep learning models for early detection of Alzheimer’s disease stage

Joint learning framework of cross-modal synthesis and diagnosis for Alzheimer’s disease by mining underlying shared modality information

Multi-Modal Data Analysis for Alzheimer’s Disease Diagnosis: An Ensemble Model Using Imagery and Genetic Features

Alzheimer’s Disease Diagnosis from Multi-Modal Data Via Feature Inductive Learning and Dual Multilevel Graph Neural Network

Multi-modal Self-paced Locality Preserving Learning for Diagnosis of Alzheimer’s Disease

Multi-modal latent space inducing ensemble SVM classifier for early dementia diagnosis with neuroimaging data

Multi-modal cross-attention network for Alzheimer's disease diagnosis with multi-modality data

Alzheimer's disease diagnosis from multi-modal data via feature inductive learning and dual multilevel graph neural network

Hypergraph-regularized multimodal learning by graph diffusion for imaging genetics based Alzheimer's Disease diagnosis

Alzheimer's disease prediction algorithm based on de-correlation constraint and multi-modal feature interaction

Trustworthy Enhanced Multi-view Multi-modal Alzheimer's Disease Prediction with Brain-wide Imaging Transcriptomics Data

Diagnosis of Alzheimer’s Disease by Canonical Correlation Analysis Based Fusion of Multi-Modal Medical Images