Abstract:Objective. Mild cognitive impairment (MCI) is a precursor to Alzheimer's disease (AD) which is an irreversible progressive neurodegenerative disease and its early diagnosis and intervention are of great significance. Recently, many deep learning methods have demonstrated the advantages of multi-modal neuroimages in MCI identification task. However, previous studies frequently simply concatenate patch-level features for prediction without modeling the dependencies among local features. Also, many methods only focus on modality-sharable information or modality-specific features and ignore their incorporation. This work aims to address above-mentioned issues and construct a model for accurate MCI identification.Approach. In this paper, we propose a multi-level fusion network for MCI identification using multi-modal neuroimages, which consists of local representation learning and dependency-aware global representation learning stages. Specifically, for each patient, we first extract multi-pair of patches from multiple same position in multi-modal neuroimages. After that, in the local representation learning stage, multiple dual-channel sub-networks, each of which consists of two modality-specific feature extraction branches and three sine-cosine fusion modules, are constructed to learn local features that preserve modality-sharable and modality specific representations simultaneously. In the dependency-aware global representation learning stage, we further capture long-range dependencies among local representations and integrate them into global ones for MCI identification.Main results. Experiments on ADNI-1/ADNI-2 datasets demonstrate the superior performance of the proposed method in MCI identification tasks (Accuracy: 0.802, sensitivity: 0.821, specificity: 0.767 in MCI diagnosis task; accuracy: 0.849, sensitivity: 0.841, specificity: 0.856 in MCI conversion task) when compared with state-of-the-art methods. The proposed classification model has demonstrated a promising potential to predict MCI conversion and identify the disease-related regions in the brain.Significance. We propose a multi-level fusion network for MCI identification using multi-modal neuroimage. The results on ADNI datasets have demonstrated its feasibility and superiority.

Fusion of Clinical Phenotypic and Multi-modal MRI for Acute Bilirubin Encephalopathy Classification

Detecting Acute Bilirubin Encephalopathy in Neonates Based on Multimodal MRI with Deep Learning

Deep Learning Network with Spatial Attention Module for Detecting Acute Bilirubin Encephalopathy in Newborns Based on Multimodal MRI

An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer's Disease Diagnosis

Feature Fused Attention Network for Acute Bilirubin Encephalopathy Classification

Enhancing Early Alzheimer's Disease Classification Accuracy Through the Fusion of Smri and Rsmeg Data: a Deep Learning Approach

A Network for Acute Bilirubin Encephalopathy Classification Based Upon Attention Mechanism and 3D Convolution Kernels

Multimodal Cross Enhanced Fusion Network for Diagnosis of Alzheimer's Disease and Subjective Memory Complaints.

BAF-Net: bidirectional attention-aware fluid pyramid feature integrated multimodal fusion network for diagnosis and prognosis

An attention-based multi-modal MRI fusion model for major depressive disorder diagnosis

Brain MR Image Classification for Alzheimer’s Disease Diagnosis Based on Multifeature Fusion

Hybrid deep spatial and statistical feature fusion for accurate MRI brain tumor classification

Deep learning and multimodal feature fusion for the aided diagnosis of Alzheimer's disease

High-Quality Fusion and Visualization for MR-PET Brain Tumor Images via Multi-Dimensional Features

Fusion of medical imaging and electronic health records with attention and multi-head machanisms

Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis

Multi-level fusion network for mild cognitive impairment identification using multi-modal neuroimages

Improved Classification Performance of EEG-fNIRS Multimodal Brain-Computer Interface Based on Multi-Domain Features and Multi-Level Progressive Learning

Multimodal cross enhanced fusion network for diagnosis of Alzheimer?s disease and

Attention-Like Multimodality Fusion With Data Augmentation for Diagnosis of Mental Disorders Using MRI

Hahn-PCNN-CNN: an end-to-end multi-modal brain medical image fusion framework useful for clinical diagnosis