Abstract:Alzheimer's disease (AD) is a neurodegenerative disorder, the most common cause of dementia, so the accurate diagnosis of AD and its prodromal stage mild cognitive impairment (MCI) is significant. Recent studies have demonstrated that multiple neuroimaging and biological measures contain complementary information for diagnosis. Many existing multi-modal models based on deep learning simply concatenate each modality's features despite substantial differences in representation spaces. In this paper, we propose a novel multi-modal cross-attention AD diagnosis (MCAD) framework to learn the interaction between modalities for better playing their complementary roles for AD diagnosis with multi-modal data including structural magnetic resonance imaging (sMRI), fluorodeoxyglucose-positron emission tomography (FDG-PET) and cerebrospinal fluid (CSF) biomarkers. Specifically, the imaging and non-imaging representations are learned by the image encoder based on cascaded dilated convolutions and CSF encoder, respectively. Then, a multi-modal interaction module is introduced, which takes advantage of cross-modal attention to integrate imaging and non-imaging information and reinforce relationships between these modalities. Moreover, an extensive objective function is designed to reduce the discrepancy between modalities for effectively fusing the features of multi-modal data, which could further improve the diagnosis performance. We evaluate the effectiveness of our proposed method on the ADNI dataset, and the extensive experiments demonstrate that our MCAD achieves superior performance for multiple AD-related classification tasks, compared to several competing methods. Also, we investigate the importance of cross-attention and the contribution of each modality to the diagnostics performance. The experimental results demonstrate that combining multi-modality data via cross-attention is helpful for accurate AD diagnosis.

A Cross-Modal Mutual Knowledge Distillation Framework for Alzheimer's Disease Diagnosis: Addressing Incomplete Modalities

MDMA: Multimodal Data and Multi-attention Based Deep Learning Model for Alzheimer’s Disease Diagnosis

Joint Learning Framework of Cross-Modal Synthesis and Diagnosis for Alzheimer's Disease by Mining Underlying Shared Modality Information.

Maximum Mean Discrepancy Based Multiple Kernel Learning for Incomplete Multimodality Neuroimaging Data

Multi-modal cross-attention network for Alzheimer's disease diagnosis with multi-modality data

Patch-based Interpretable Deep Learning Framework for Alzheimer’s Disease Diagnosis Using Multimodal Data

Multimodal Attention-based Deep Learning for Alzheimer's Disease Diagnosis

Multimodal 3D Deep Learning for Early Diagnosis of Alzheimer's Disease

A Multi-classification Accessment Framework for Reproducible Evaluation of Multimodal Learning in Alzheimer's Disease

Recognition of Alzheimer's disease and Mild Cognitive Impairment with multimodal image-derived biomarkers and Multiple Kernel Learning

Multi-scale multimodal deep learning framework for Alzheimer's disease diagnosis

Patch-based Deep Multi-Modal Learning Framework for Alzheimer’s Disease Diagnosis Using Multi-View Neuroimaging

Multimodal deep learning models for early detection of Alzheimer’s disease stage

Deep Multi-Modal Discriminative and Interpretability Network for Alzheimer's Disease Diagnosis

Joint learning framework of cross-modal synthesis and diagnosis for Alzheimer’s disease by mining underlying shared modality information

Disentangle First, then Distill: A Unified Framework for Missing Modality Imputation and Alzheimer's Disease Diagnosis.

Toward Robust Early Detection of Alzheimer's Disease via an Integrated Multimodal Learning Approach

Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer's Disease using structural MR and FDG-PET images

Multi-modal Self-paced Locality Preserving Learning for Diagnosis of Alzheimer’s Disease

Alzheimer’s Disease Diagnosis Via Specific-Shared Representation Learning in Multimodal Neuroimaging

ADMultiImg: a Novel Missing Modality Transfer Learning Based CAD System for Diagnosis of MCI Due to AD Using Incomplete Multi-Modality Imaging Data.