Abstract:Objective. Mild cognitive impairment (MCI) is a precursor to Alzheimer's disease (AD) which is an irreversible progressive neurodegenerative disease and its early diagnosis and intervention are of great significance. Recently, many deep learning methods have demonstrated the advantages of multi-modal neuroimages in MCI identification task. However, previous studies frequently simply concatenate patch-level features for prediction without modeling the dependencies among local features. Also, many methods only focus on modality-sharable information or modality-specific features and ignore their incorporation. This work aims to address above-mentioned issues and construct a model for accurate MCI identification.Approach. In this paper, we propose a multi-level fusion network for MCI identification using multi-modal neuroimages, which consists of local representation learning and dependency-aware global representation learning stages. Specifically, for each patient, we first extract multi-pair of patches from multiple same position in multi-modal neuroimages. After that, in the local representation learning stage, multiple dual-channel sub-networks, each of which consists of two modality-specific feature extraction branches and three sine-cosine fusion modules, are constructed to learn local features that preserve modality-sharable and modality specific representations simultaneously. In the dependency-aware global representation learning stage, we further capture long-range dependencies among local representations and integrate them into global ones for MCI identification.Main results. Experiments on ADNI-1/ADNI-2 datasets demonstrate the superior performance of the proposed method in MCI identification tasks (Accuracy: 0.802, sensitivity: 0.821, specificity: 0.767 in MCI diagnosis task; accuracy: 0.849, sensitivity: 0.841, specificity: 0.856 in MCI conversion task) when compared with state-of-the-art methods. The proposed classification model has demonstrated a promising potential to predict MCI conversion and identify the disease-related regions in the brain.Significance. We propose a multi-level fusion network for MCI identification using multi-modal neuroimage. The results on ADNI datasets have demonstrated its feasibility and superiority.

MMF-NNs: Multi-modal Multi-granularity Fusion Neural Networks for brain networks and its application to epilepsy identification

MSTGC: Multi-Channel Spatio-Temporal Graph Convolution Network for Multi-Modal Brain Networks Fusion

Multi-modal feature fusion with multi-head self-attention for epileptic EEG signals

Incomplete multi-modal brain image fusion for epilepsy classification

Multimodal Brain Network Jointly Construction and Fusion for Diagnosis of Epilepsy

Coherent Pattern in Multi-Layer Brain Networks: Application to Epilepsy Identification

MM-BiFPN: Multi-Modality Fusion Network With Bi-FPN for MRI Brain Tumor Segmentation

Multi feature fusion network for schizophrenia classification and abnormal brain network recognition

Multimodal Triplet Attention Network for Brain Disease Diagnosis

A Temporal Multi-view Fuzzy Classifier for Fusion Identification on Epileptic Brain Network

Multi-graph Fusion for Functional Neuroimaging Biomarker Detection

MFD-Net: Modality Fusion Diffractive Network for Segmentation of Multimodal Brain Tumor Image

A Heterogeneous Graph Based Framework for Multimodal Neuroimaging Fusion Learning

Multi-modality Hierarchical Fusion Network for Lumbar Spine Segmentation with Magnetic Resonance Images

Modality-level cross-connection and attentional feature fusion based deep neural network for multi-modal brain tumor segmentation

MLMFNet: A Multi-Level Modality Fusion Network for Multi-Modal Accelerated MRI Reconstruction

Multimodal Fusion of Brain Imaging Data: Methods and Applications

Multi-level fusion network for mild cognitive impairment identification using multi-modal neuroimages

MM-Net: A MixFormer-Based Multi-Scale Network for Anatomical and Functional Image Fusion

Persistent Feature Analysis of Multimodal Brain Networks Using Generalized Fused Lasso for EMCI Identification

Brain tumor segmentation by combining MultiEncoder UNet with wavelet fusion