WISE 2014 Challenge: Multi-label Classification of Print Media Articles to Topics

Grigorios Tsoumakas,A. Papadopoulos,Weining Qian,S. Vologiannidis,A. D'yakonov,Antti Puurula,Jesse Read,J. Svec,Stanislav Semenov

DOI: https://doi.org/10.1007/978-3-319-11746-1_40

2014-10-12

Abstract:

What problem does this paper attempt to address?

Multimodal classification of Alzheimer's disease and mild cognitive impairment

Daoqiang Zhang,Yaping Wang,Luping Zhou,Hong Yuan,Dinggang Shen,the Alzheimer's Disease Neuroimaging Initiative

DOI: https://doi.org/10.1016/j.neuroimage.2011.01.008

IF: 5.7

2011-04-01

NeuroImage

Abstract:Effective and accurate diagnosis of Alzheimer's disease (AD), as well as its prodromal stage (i.e., mild cognitive impairment (MCI)), has attracted more and more attention recently. So far, multiple biomarkers have been shown to be sensitive to the diagnosis of AD and MCI, i.e., structural MR imaging (MRI) for brain atrophy measurement, functional imaging (e.g., FDG-PET) for hypometabolism quantification, and cerebrospinal fluid (CSF) for quantification of specific proteins. However, most existing research focuses on only a single modality of biomarkers for diagnosis of AD and MCI, although recent studies have shown that different biomarkers may provide complementary information for the diagnosis of AD and MCI. In this paper, we propose to combine three modalities of biomarkers, i.e., MRI, FDG-PET, and CSF biomarkers, to discriminate between AD (or MCI) and healthy controls, using a kernel combination method. Specifically, ADNI baseline MRI, FDG-PET, and CSF data from 51AD patients, 99 MCI patients (including 43 MCI converters who had converted to AD within 18 months and 56 MCI non-converters who had not converted to AD within 18 months), and 52 healthy controls are used for development and validation of our proposed multimodal classification method. In particular, for each MR or FDG-PET image, 93 volumetric features are extracted from the 93 regions of interest (ROIs), automatically labeled by an atlas warping algorithm. For CSF biomarkers, their original values are directly used as features. Then, a linear support vector machine (SVM) is adopted to evaluate the classification accuracy, using a 10-fold cross-validation. As a result, for classifying AD from healthy controls, we achieve a classification accuracy of 93.2% (with a sensitivity of 93% and a specificity of 93.3%) when combining all three modalities of biomarkers, and only 86.5% when using even the best individual modality of biomarkers. Similarly, for classifying MCI from healthy controls, we achieve a classification accuracy of 76.4% (with a sensitivity of 81.8% and a specificity of 66%) for our combined method, and only 72% even using the best individual modality of biomarkers. Further analysis on MCI sensitivity of our combined method indicates that 91.5% of MCI converters and 73.4% of MCI non-converters are correctly classified. Moreover, we also evaluate the classification performance when employing a feature selection method to select the most discriminative MR and FDG-PET features. Again, our combined method shows considerably better performance, compared to the case of using an individual modality of biomarkers.

radiology, nuclear medicine & medical imaging,neurosciences,neuroimaging
ASMFS: Adaptive-Similarity-based Multi-modality Feature Selection for Classification of Alzheimer's Disease

Yuang Shi,Chen Zu,Mei Hong,Luping Zhou,Lei Wang,Xi Wu,Jiliu Zhou,Daoqiang Zhang,Yan Wang

DOI: https://doi.org/10.1016/j.patcog.2022.108566

IF: 8

2022-01-01

Pattern Recognition

Abstract:Multimodal classification methods using different modalities have great advantages over traditional single-modality-based ones for the diagnosis of Alzheimer's disease (AD) and its prodromal stage mild cognitive impairment (MCI). With the increasing amount of high-dimensional heterogeneous data to be processed, multi-modality feature selection has become a crucial research direction for AD classification. However, traditional methods usually depict the data structure using pre-defined similarity matrix as a priori, which is difficult to precisely measure the intrinsic relationship across different modalities in high-dimensional space. In this paper, we propose a novel multimodal feature selection method called Adaptive-Similarity-based Multi-modality Feature Selection (ASMFS) which performs adaptive similarity learning and feature selection simultaneously. Specifically, a similarity matrix is learned by jointly considering different modalities and at the same time, an efficient feature selection is conducted by imposing group sparsity-inducing l 2 , 1-norm constraint. Evaluated on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database with baseline MRI and FDG-PET imaging data collected from 51 AD, 43 MCI converters (MCI-C), 56 MCI non-converters (MCI-NC) and 52 normal controls (NC), we demonstrate the effectiveness and superiority of our proposed method against other state-of-the-art approaches for multi modality classification of AD/MCI. (c) 2022 Elsevier Ltd. All rights reserved.
Multi-modal Self-paced Locality Preserving Learning for Diagnosis of Alzheimer’s Disease

Xiaoke Hao,Ruxue Wang,Yingchun Guo,Yunjia Xiao,Ming Yu,Meiling Wang,Weibin Chen,Daoqiang Zhang,Alzheimer's Disease Neuroimaging Initiative

DOI: https://doi.org/10.1109/tcds.2022.3189701

IF: 4.546

2022-01-01

IEEE Transactions on Cognitive and Developmental Systems

Abstract:Alzheimer’s disease (AD) is an irreversible neurodegenerative disease that severely impairs human thinking and memory. The accurate diagnosis of AD and its prodromal stages, such as mild cognitive impairment (MCI), is very important for timely treatment or possible interventions of AD. Recent studies have shown that multiple neuroimaging and biological measures contain supplementary information for diagnosis and prognosis. Most existing methods are proposed to simply integrate the multimodal data and train the model using all samples once, which do not fully explore the structural information across the different modalities and ignore the significance of sample learning in the training process. In this article, we propose a multimodal self-paced locality-preserving learning (MSLPL) framework to preserve the inherent structural relationships of the original data and realize the sample selection process from “simple” to “complex.” Specifically, the model can project the neuroimaging and genetic data into the label space and learn dimensionality reduction manners with preserving locality structure. Meanwhile, the contributions of each sample are adaptively evaluated by weighting optimization so that the impact of noises can be reduced during model training by self-paced learning (SPL). Finally, a multikernel support vector machine (MK-SVM) is used to fuse the features selected from different modalities for the final prediction. We evaluate MSLPL on 913 subjects from the AD neuroimaging initiative (ADNI) database with imaging and genetic data. The experimental results demonstrate that the proposed method can achieve better classification performances compared with the start-of-the-art multimodality-based methods.

robotics,computer science, artificial intelligence,neurosciences
Multi-modal cross-attention network for Alzheimer's disease diagnosis with multi-modality data

Jin Zhang,Xiaohai He,Yan Liu,Qingyan Cai,Honggang Chen,Linbo Qing

DOI: https://doi.org/10.1016/j.compbiomed.2023.107050

IF: 7.7

2023-05-24

Computers in Biology and Medicine

Abstract:Alzheimer's disease (AD) is a neurodegenerative disorder, the most common cause of dementia, so the accurate diagnosis of AD and its prodromal stage mild cognitive impairment (MCI) is significant. Recent studies have demonstrated that multiple neuroimaging and biological measures contain complementary information for diagnosis. Many existing multi-modal models based on deep learning simply concatenate each modality's features despite substantial differences in representation spaces. In this paper, we propose a novel multi-modal cross-attention AD diagnosis (MCAD) framework to learn the interaction between modalities for better playing their complementary roles for AD diagnosis with multi-modal data including structural magnetic resonance imaging (sMRI), fluorodeoxyglucose-positron emission tomography (FDG-PET) and cerebrospinal fluid (CSF) biomarkers. Specifically, the imaging and non-imaging representations are learned by the image encoder based on cascaded dilated convolutions and CSF encoder, respectively. Then, a multi-modal interaction module is introduced, which takes advantage of cross-modal attention to integrate imaging and non-imaging information and reinforce relationships between these modalities. Moreover, an extensive objective function is designed to reduce the discrepancy between modalities for effectively fusing the features of multi-modal data, which could further improve the diagnosis performance. We evaluate the effectiveness of our proposed method on the ADNI dataset, and the extensive experiments demonstrate that our MCAD achieves superior performance for multiple AD-related classification tasks, compared to several competing methods. Also, we investigate the importance of cross-attention and the contribution of each modality to the diagnostics performance. The experimental results demonstrate that combining multi-modality data via cross-attention is helpful for accurate AD diagnosis.

engineering, biomedical,computer science, interdisciplinary applications,mathematical & computational biology,biology
Multi-modal latent space inducing ensemble SVM classifier for early dementia diagnosis with neuroimaging data

Tao Zhou,Kim-Han Thung,Mingxia Liu,Feng Shi,Changqing Zhang,Dinggang Shen

DOI: https://doi.org/10.1016/j.media.2019.101630

Abstract:Fusing multi-modality data is crucial for accurate identification of brain disorder as different modalities can provide complementary perspectives of complex neurodegenerative disease. However, there are at least four common issues associated with the existing fusion methods. First, many existing fusion methods simply concatenate features from each modality without considering the correlations among different modalities. Second, most existing methods often make prediction based on a single classifier, which might not be able to address the heterogeneity of the Alzheimer's disease (AD) progression. Third, many existing methods often employ feature selection (or reduction) and classifier training in two independent steps, without considering the fact that the two pipelined steps are highly related to each other. Forth, there are missing neuroimaging data for some of the participants (e.g., missing PET data), due to the participants' "no-show" or dropout. In this paper, to address the above issues, we propose an early AD diagnosis framework via novel multi-modality latent space inducing ensemble SVM classifier. Specifically, we first project the neuroimaging data from different modalities into a latent space, and then map the learned latent representations into the label space to learn multiple diversified classifiers. Finally, we obtain the more reliable classification results by using an ensemble strategy. More importantly, we present a Complete Multi-modality Latent Space (CMLS) learning model for complete multi-modality data and also an Incomplete Multi-modality Latent Space (IMLS) learning model for incomplete multi-modality data. Extensive experiments using the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset have demonstrated that our proposed models outperform other state-of-the-art methods.
Alzheimer’s Disease Diagnosis from Multi-Modal Data Via Feature Inductive Learning and Dual Multilevel Graph Neural Network

Baiying Lei,Yafeng Li,Wanyi Fu,Peng Yang,Shaobin Chen,Tianfu Wang,Xiaohua Xiao,Tianye Niu,Yu Fu,Shuqiang Wang,Hongbin Han,Jing Qin

DOI: https://doi.org/10.1016/j.media.2024.103213

IF: 10.9

2024-01-01

Medical Image Analysis

Abstract:Multi-modal data can provide complementary information of Alzheimer’s disease (AD) and its development from different perspectives. Such information is closely related to the diagnosis, prevention, and treatment of AD, and hence it is necessary and critical to study AD through multi-modal data. Existing learning methods, however, usually ignore the influence of feature heterogeneity and directly fuse features in the last stages. Furthermore, most of these methods only focus on local fusion features or global fusion features, neglecting the complementariness of features at different levels and thus not sufficiently leveraging information embedded in multi-modal data. To overcome these shortcomings, we propose a novel framework for AD diagnosis that fuses gene, imaging, protein, and clinical data. Our framework learns feature representations under the same feature space for different modalities through a feature induction learning (FIL) module, thereby alleviating the impact of feature heterogeneity. Furthermore, in our framework, local and global salient multi-modal feature interaction information at different levels is extracted through a novel dual multilevel graph neural network (DMGNN). We extensively validate the proposed method on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset and experimental results demonstrate our method consistently outperforms other state-of-the-art multi-modal fusion methods. The code is publicly available on the GitHub website. (https://github.com/xiankantingqianxue/MIA-code.git)
Alzheimer's disease diagnosis from multi-modal data via feature inductive learning and dual multilevel graph neural network

Baiying Lei,Yafeng Li,Wanyi Fu,Peng Yang,Shaobin Chen,Tianfu Wang,Xiaohua Xiao,Tianye Niu,Yu Fu,Shuqiang Wang,Hongbin Han,Jing Qin

DOI: https://doi.org/10.1016/j.media.2024.103213

IF: 10.9

2024-05-30

Medical Image Analysis

Abstract:Multi-modal data can provide complementary information of Alzheimer's disease (AD) and its development from different perspectives. Such information is closely related to the diagnosis, prevention, and treatment of AD, and hence it is necessary and critical to study AD through multi-modal data. Existing learning methods, however, usually ignore the influence of feature heterogeneity and directly fuse features in the last stages. Furthermore, most of these methods only focus on local fusion features or global fusion features, neglecting the complementariness of features at different levels and thus not sufficiently leveraging information embedded in multi-modal data. To overcome these shortcomings, we propose a novel framework for AD diagnosis that fuses gene, imaging, protein, and clinical data. Our framework learns feature representations under the same feature space for different modalities through a feature induction learning (FIL) module, thereby alleviating the impact of feature heterogeneity. Furthermore, in our framework, local and global salient multi-modal feature interaction information at different levels is extracted through a novel dual multilevel graph neural network (DMGNN). We extensively validate the proposed method on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset and experimental results demonstrate our method consistently outperforms other state-of-the-art multi-modal fusion methods. The code is publicly available on the GitHub website. ( https://github.com/xiankantingqianxue/MIA-code.git )

engineering, biomedical,computer science, interdisciplinary applications, artificial intelligence,radiology, nuclear medicine & medical imaging
Multi-modality sparse representation-based classification for Alzheimer's disease and mild cognitive impairment

Lele Xu,Xia Wu,Kewei Chen,Li Yao

DOI: https://doi.org/10.1016/j.cmpb.2015.08.004

Abstract:Background and objective: The discrimination of Alzheimer's disease (AD) and its prodromal stage known as mild cognitive impairment (MCI) from normal control (NC) is important for patients' timely treatment. The simultaneous use of multi-modality data has been demonstrated to be helpful for more accurate identification. The current study focused on extending a multi-modality algorithm and evaluating the method by identifying AD/MCI. Methods: In this study, sparse representation-based classification (SRC), a well-developed method in pattern recognition and machine learning, was extended to a multi-modality classification framework named as weighted multi-modality SRC (wmSRC). Data including three modalities of volumetric magnetic resonance imaging (MRI), fluorodeoxyglucose (FDG) positron emission tomography (PET) and florbetapir PET from the Alzheimer's disease Neuroimaging Initiative database were adopted for AD/MCI classification (113 AD patients, 110 MCI patients and 117 NC subjects). Results: Adopting wmSRC, the classification accuracy achieved 94.8% for AD vs. NC, 74.5% for MCI vs. NC, and 77.8% for progressive MCI vs. stable MCI, superior to or comparable with the results of some other state-of-the-art models in recent multi-modality researches. Conclusions: The wmSRC method is a promising tool for classification with multi-modality data. It could be effective for identifying diseases from NC with neuroimaging data, which could be helpful for the timely diagnosis and treatment of diseases.
Recognition of Alzheimer's disease and Mild Cognitive Impairment with multimodal image-derived biomarkers and Multiple Kernel Learning

Olfa Ben Ahmed,Jenny Benois-Pineau,Michelle Allard,Gwénaëlle Catheline,Chokri Ben Amar,For the Alzheimer's Disease Neuroimaging Initiative

DOI: https://doi.org/10.1016/j.neucom.2016.08.041

IF: 6

2017-01-01

Neurocomputing

Abstract:Computer-Aided Diagnosis (CAD) of Alzheimer's disease (AD) has drawn the attention of computer vision research community over the last few years. Several attempts have been made to adapt pattern recognition approaches to specific neuroimaging data such as Structural MRI (sMRI) for early AD diagnosis. One strategy is to boost the discrimination power of such approaches by integrating complementary imaging modalities in a single learning framework. Diffusion Tensor Imaging (DTI) is a new and promising modality giving complementary information to the anatomical MRI. However, including relevant DTI information from such modality is a challenging problem. In this paper, we propose to extract local image-derived biomarkers from DTI and sMRI to construct multimodal AD signatures. To assess the relevance of such modalities as well as to optimize the classifier, we integrate complementary information using a Multiple Kernel Learning (MKL) framework for AD subjects recognition. To evaluate our method, we perform experiments on a subset from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Both T1-weighted MRI and Mean Diffusivity (MD) maps from the DTI modality of 45 AD patients, 52 Normal Control (NC) and 58 Mild Cognitive Impairment (MCI) subjects have been used. The obtained results indicate that our multimodal approach yields significant improvement in accuracy over using each single modality independently. The classification accuracies obtained by the proposed method are 90.2%, 79.42% and 76.63% for respectively AD vs. NC, MCI vs. NC and AD vs. MCI binary classification problems. For the MCI classification problem, the proposed fusion framework leads to an average increase about at least 9% for the accuracy, 5% for the specificity and 15% for the sensitivity.

computer science, artificial intelligence
Multi-modal discriminative dictionary learning for Alzheimer's disease and mild cognitive impairment

Qing Li,Xia Wu,Lele Xu,Kewei Chen,Li Yao,Rui Li

DOI: https://doi.org/10.1016/j.cmpb.2017.07.003

Abstract:Background and objective: The differentiation of mild cognitive impairment (MCI), which is the prodromal stage of Alzheimer's disease (AD), from normal control (NC) is important as the recent research emphasis on early pre-clinical stage for possible disease abnormality identification, intervention and even possible prevention. Methods: The current study puts forward a multi-modal supervised within-class-similarity discriminative dictionary learning algorithm (SCDDL) we introduced previously for distinguishing MCI from NC. The proposed new algorithm was based on weighted combination and named as multi-modality SCDDL (mSCDDL). Structural magnetic resonance imaging (sMRI), fluorodeoxyglucose (FDG) positron emission tomography (PET) and florbetapir PET data of 113 AD patients, 110 MCI patients and 117 NC subjects from the Alzheimer's disease Neuroimaging Initiative database were adopted for classification between MCI and NC, as well as between AD and NC. Results: Adopting mSCDDL, the classification accuracy achieved 98.5% for AD vs. NC and 82.8% for MCI vs. NC, which were superior to or comparable with the results of some other state-of-the-art approaches as reported in recent multi-modality publications. Conclusions: The mSCDDL procedure was a promising tool in assisting early diseases diagnosis using neuroimaging data.
Enhanced Multimodal Low-rank Embedding based Feature Selection Model for Multimodal Alzheimer's Disease Diagnosis

Zhi Chen,Yongguo Liu,Yun Zhang,Jiajing Zhu,Qiaoqin Li,Xindong Wu

DOI: https://doi.org/10.1109/TMI.2024.3464861

2024-09-20

Abstract:Identification of Alzheimer's disease (AD) with multimodal neuroimaging data has been receiving increasing attention. However, the presence of numerous redundant features and corrupted neuroimages within multimodal datasets poses significant challenges for existing methods. In this paper, we propose a feature selection method named Enhanced Multimodal Low-rank Embedding (EMLE) for multimodal AD diagnosis. Unlike previous methods utilizing convex relaxations of the ℓ2,0-norm, EMLE exploits an ℓ2,γ-norm regularized projection matrix to obtain an embedding representation and select informative features jointly for each modality. The ℓ2,γ-norm, employing an upper-bounded nonconvex Minimax Concave Penalty (MCP) function to characterize sparsity, offers a superior approximation for the ℓ2,0-norm compared to other convex relaxations. Next, a similarity graph is learned based on the self-expressiveness property to increase the robustness to corrupted data. As the approximation coefficient vectors of samples from the same class should be highly correlated, an MCP function introduced norm, i.e., matrix γ-norm, is applied to constrain the rank of the graph. Furthermore, recognizing that diverse modalities should share an underlying structure related to AD, we establish a consensus graph for all modalities to unveil intrinsic structures across multiple modalities. Finally, we fuse the embedding representations of all modalities into the label space to incorporate supervisory information. The results of extensive experiments on the Alzheimer's Disease Neuroimaging Initiative datasets verify the discriminability of the features selected by EMLE.
Double-attention Assisted Multi-task Learning for the Alzheimer’s Disease Prediction from Mild Cognitive Impairment*

Yiran Xu,Longfei Ma,Hui Zhang,Hongen Liao

DOI: https://doi.org/10.1109/embc40787.2023.10340815

2023-01-01

Abstract:Alzheimer’s disease (AD) is a progressive neurode-generative disease. Identifying the mild cognitive impairment (MCI) subjects who will convert to AD is essential for early intervention to slow the irreversible brain damage and cognitive decline. In this paper, we propose a novel double-attention assisted multi-task framework for the MCI conversion prediction task. By introducing an auxiliary grey matter segmentation task along with an adaptive dynamic weight average strategy to balance the impact of each task. Then, a double-attention module is incorporated to leverage both the classification and the segmentation attention information to guide the network to focus more on the structural alteration regions for better discrimination of AD pathology, as well as increase the interpretability of the network. Extensive experiments on a publicly available dataset demonstrate that the proposed method significantly outperforms the approaches using the same image modality.
A hybrid multimodal machine learning model for Detecting Alzheimer's disease

Jinhua Sheng,Qian Zhang,Qiao Zhang,Luyun Wang,Ze Yang,Yu Xin,Binbing Wang

DOI: https://doi.org/10.1016/j.compbiomed.2024.108035

IF: 7.7

2024-03-01

Computers in Biology and Medicine

Abstract:Alzheimer's disease (AD) diagnosis utilizing single modality neuroimaging data has limitations. Multimodal fusion of complementary biomarkers may improve diagnostic performance. This study proposes a multimodal machine learning framework integrating magnetic resonance imaging (MRI), positron emission tomography (PET) and cerebrospinal fluid (CSF) assays for enhanced AD characterization. The model incorporates a hybrid algorithm combining enhanced Harris Hawks Optimization (HHO) algorithm referred to as ILHHO, with Kernel Extreme Learning Machine (KELM) classifier for simultaneous feature selection and classification. ILHHO enhances HHO's search efficiency by integrating iterative mapping (IM) to improve population diversity and local escaping operator (LEO) to balance exploration-exploitation. Comparative analysis with other improved HHO algorithms, classic meta-heuristic algorithms (MHAs), and state-of-the-art MHAs on IEEE CEC2014 benchmark functions indicates that ILHHO achieves superior optimization performance compared to other comparative algorithms. The synergistic ILHHO-KELM model is evaluated on 202 AD Neuroimaging Initiative (ADNI) subjects. Results demonstrate superior multimodal classification accuracy over single modalities, validating the importance of fusing heterogeneous biomarkers. MRI + PET + CSF achieves 99.2 % accuracy for AD vs. normal control (NC), outperforming conventional and proposed methods. Discriminative feature analysis provides further insights into differential AD-related neurodegeneration patterns detected by MRI and PET. The differential PET and MRI features demonstrate how the two modalities provide complementary biomarkers. The neuroanatomical relevance of selected features supports ILHHO-KELM's potential for extracting sensitive AD imaging signatures. Overall, the study showcases the advantages of capitalizing on complementary multimodal data through advanced feature learning techniques for improving AD diagnosis.

engineering, biomedical,computer science, interdisciplinary applications,mathematical & computational biology,biology
A feature-aware multimodal framework with auto-fusion for Alzheimer's disease diagnosis

Meiwei Zhang,Qiushi Cui,Yang Lü,Wenyuan Li

DOI: https://doi.org/10.1016/j.compbiomed.2024.108740

Abstract:Alzheimer's disease (AD), one of the most common dementias, has about 4.6 million new cases yearly worldwide. Due to the significant amount of suspected AD patients, early screening for the disease has become particularly important. There are diversified types of AD diagnosis data, such as cognitive tests, images, and risk factors, many prior investigations have primarily concentrated on integrating only high-dimensional features and simple fusion concatenation, resulting in less-than-optimal outcomes for AD diagnosis. Therefore, We propose an enhanced multimodal AD diagnostic framework comprising a feature-aware module and an automatic model fusion strategy (AMFS). To preserve the correlation and significance features within a low-dimensional space, the feature-aware module employs a low-dimensional SHapley Additive exPlanation (SHAP) boosting feature selection as the initial step, following this analysis, diverse tiers of low-dimensional features are extracted from patients' biological data. Besides, in the high-dimensional stage, the feature-aware module integrates cross-modal attention mechanisms to capture subtle relationships among different cognitive domains, neuroimaging modalities, and risk factors. Subsequently, we integrate the aforementioned feature-aware module with graph convolutional networks (GCN) to address heterogeneous data in multimodal AD, while also possessing the capability to perceive relationships between different modalities. Lastly, our proposed AMFS autonomously learns optimal parameters for aligning two sub-models. The validation tests using two ADNI datasets show the high accuracies of 95.9% and 91.9% respectively, in AD diagnosis. The methods efficiently select features from multimodal AD data, optimizing model fusion for potential clinical assistance in diagnostics.
An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer's Disease Diagnosis

Juan Song,Jian Zheng,Ping Li,Xiaoyuan Lu,Guangming Zhu,Peiyi Shen

DOI: https://doi.org/10.3389/fdgth.2021.637386

2021-02-26

Frontiers in Digital Health

Abstract:Alzheimer's disease (AD) is an irreversible brain disease that severely damages human thinking and memory. Early diagnosis plays an important part in the prevention and treatment of AD. Neuroimaging-based computer-aided diagnosis (CAD) has shown that deep learning methods using multimodal images are beneficial to guide AD detection. In recent years, many methods based on multimodal feature learning have been proposed to extract and fuse latent representation information from different neuroimaging modalities including magnetic resonance imaging (MRI) and 18-fluorodeoxyglucose positron emission tomography (FDG-PET). However, these methods lack the interpretability required to clearly explain the specific meaning of the extracted information. To make the multimodal fusion process more persuasive, we propose an image fusion method to aid AD diagnosis. Specifically, we fuse the gray matter (GM) tissue area of brain MRI and FDG-PET images by registration and mask coding to obtain a new fused modality called “GM-PET.” The resulting single composite image emphasizes the GM area that is critical for AD diagnosis, while retaining both the contour and metabolic characteristics of the subject's brain tissue. In addition, we use the three-dimensional simple convolutional neural network (3D Simple CNN) and 3D Multi-Scale CNN to evaluate the effectiveness of our image fusion method in binary classification and multi-classification tasks. Experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset indicate that the proposed image fusion method achieves better overall performance than unimodal and feature fusion methods, and that it outperforms state-of-the-art methods for AD diagnosis.
A Multi-Modal Classification Method for Early Diagnosis of Mild Cognitive Impairment and Alzheimer's Disease Using Three Paradigms With Various Task Difficulties

Sheng Chen,Chutian Zhang,Hongjun Yang,Liang Peng,Haiqun Xie,Zeping Lv,Zeng-Guang Hou

DOI: https://doi.org/10.1109/tnsre.2024.3379891

IF: 4.9

2024-04-06

IEEE Transactions on Neural Systems and Rehabilitation Engineering

Abstract:Alzheimer's Disease (AD) accounts for the majority of dementia, and Mild Cognitive Impairment (MCI) is the early stage of AD. Early and accurate diagnosis of dementia plays a vital role in more targeted treatments and effectively halting disease progression. However, the clinical diagnosis of dementia requires various examinations, which are expensive and require a high level of expertise from the doctor. In this paper, we proposed a classification method based on multi-modal data including Electroencephalogram (EEG), eye tracking and behavioral data for early diagnosis of AD and MCI. Paradigms with various task difficulties were used to identify different severity of dementia: eye movement task and resting-state EEG tasks were used to detect AD, while eye movement task and delayed match-to-sample task were used to detect MCI. Besides, the effects of different features were compared and suitable EEG channels were selected for the detection. Furthermore, we proposed a data augmentation method to enlarge the dataset, designed an extra ERPNet feature extract layer to extract multi-modal features and used domain-adversarial neural network to improve the performance of MCI diagnosis. We achieved an average accuracy of 88.81% for MCI diagnosis and 100% for AD diagnosis. The results of this paper suggest that our classification method can provide a feasible and affordable way to diagnose dementia.

engineering, biomedical,rehabilitation
Self-paced Semi-Supervised Feature Selection with Application to Multi-Modal Alzheimer’s Disease Classification

Chao Zhang,Wentao Fan,Bo Wang,Chunlin Chen,Huaxiong Li

DOI: https://doi.org/10.1016/j.inffus.2024.102345

IF: 18.6

2024-01-01

Information Fusion

Abstract:Semi-supervised multi-modal learning has attracted much attention due to the expense and scarcity of data labels, especially in disease diagnosis field. Most existing methods follow the paradigm by iteratively inferring the pseudo-labels of unlabeled data and add them into training sequence, but they ignore the reliability of those pseudo-labels, where inaccurate and wrong supervision will lead to negative influence on model learning. In this paper, we propose a Self-paced Semi-supervised Multi-modal Feature Selection (SSMFS) method, and apply it to Alzheimer’s disease classification. Specifically, SSMFS projects multi-modal biomedical data into the common label space with discriminative feature selection. Under the guidance of prior multi-modal similarity graphs, a unified graph is adaptively learned and embedded to preserve the neighborhood structures. More importantly, SSMFS dynamically investigates the discriminability and credibility of pseudo-labels, and adaptively assigns a weight to each unlabeled sample via self-paced learning such that the negative influence of wrong supervision can be reduced. Finally, a multi-kernel support vector machine is used to fuse the selected multi-modal features for final disease prediction. Experimental results on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset demonstrate the effectiveness of our method.
Deep learning and multimodal feature fusion for the aided diagnosis of Alzheimer's disease

Jia, Hongfei,Lao, Huan

DOI: https://doi.org/10.1007/s00521-022-07501-0

2022-07-18

Neural Computing and Applications

Abstract:The accurate diagnosis of Alzheimer’s disease (AD) in the early stages, such as significant memory concern (SMC) and mild cognitive impairment (MCI), is essential in order to slow its progression through timely treatment. Recent achievements have shown that fusing multimodal neuroimaging data effectively facilitates AD diagnosis. However, most proposed fusion methods simply add or concatenate multimodal features and do not make full use of nonlinear features and texture features across the range of modalities. This paper proposes a diagnostic model that effectively diagnoses AD in different stages by fusing functional magnetic resonance imaging (fMRI) and structural MRI (sMRI) information. First, fMRI and sMRI scans are preprocessed, and mean regional homogeneity (mReHo) transformation is performed for the preprocessed fMRI scans. Then, 3DMR-PCANet extracts features of mReHo images. The basic ResNet module is stacked to build a 3DResNet-10 model for feature extraction of sMRI scans. Next, two image features are fused by kernel canonical correlation analysis. Finally, a support vector machine (SVM) is utilized for the classification of fused features. Experimental results on the Alzheimer's Disease Neuroimaging dataset demonstrate the effectiveness of the proposed method. Specifically, this method improves on the accuracy, specificity, sensitivity, F1 value and area under the curve (AUC) of existing methods in comparisons of the normal control (NC) versus SMC, NC versus MCI, NC versus AD, SMC versus MCI, SMC versus AD, and MCI versus AD groups, which confirms that the proposed method can mine information on the correlation between fMRI and sMRI data of the same subject and can effectively classify AD patients in different stages.

computer science, artificial intelligence
Multimodal Identification of Alzheimer's Disease: A Review

Guian Fang,Mengsha Liu,Yi Zhong,Zhuolin Zhang,Jiehui Huang,Zhenchao Tang,Calvin Yu-Chian Chen

2023-10-06

Abstract:Alzheimer's disease is a progressive neurological disorder characterized by cognitive impairment and memory loss. With the increasing aging population, the incidence of AD is continuously rising, making early diagnosis and intervention an urgent need. In recent years, a considerable number of teams have applied computer-aided diagnostic techniques to early classification research of AD. Most studies have utilized imaging modalities such as magnetic resonance imaging (MRI), positron emission tomography (PET), and electroencephalogram (EEG). However, there have also been studies that attempted to use other modalities as input features for the models, such as sound, posture, biomarkers, cognitive assessment scores, and their fusion. Experimental results have shown that the combination of multiple modalities often leads to better performance compared to a single modality. Therefore, this paper will focus on different modalities and their fusion, thoroughly elucidate the mechanisms of various modalities, explore which methods should be combined to better harness their utility, analyze and summarize the literature in the field of early classification of AD in recent years, in order to explore more possibilities of modality combinations.

Image and Video Processing,Computer Vision and Pattern Recognition
Alzheimer's disease prediction algorithm based on de-correlation constraint and multi-modal feature interaction

Jiayuan Cheng,Huabin Wang,Shicheng Wei,Jiahao Mei,Fei Liu,Gong Zhang

DOI: https://doi.org/10.1016/j.compbiomed.2024.108000

IF: 7.7

2024-01-18

Computers in Biology and Medicine

Abstract:Alzheimer's disease (AD) is a neurodegenerative disease characterized by various pathological changes. Utilizing multimodal data from Fluorodeoxyglucose positron emission tomography(FDG-PET) and Magnetic Resonance Imaging(MRI) of the brain can offer comprehensive information about the lesions from different perspectives and improve the accuracy of prediction. However, there are significant differences in the feature space of multimodal data. Commonly, the simple concatenation of multimodal features can cause the model to struggle in distinguishing and utilizing the complementary information between different modalities, thus affecting the accuracy of predictions. Therefore, we propose an AD prediction model based on de-correlation constraint and multi-modal feature interaction. This model consists of the following three parts: (1) The feature extractor employs residual connections and attention mechanisms to capture distinctive lesion features from FDG-PET and MRI data within their respective modalities. (2) The de-correlation constraint function enhances the model's capacity to extract complementary information from different modalities by reducing the feature similarity between them. (3) The mutual attention feature fusion module interacts with the features within and between modalities to enhance the modal-specific features and adaptively adjust the weights of these features based on information from other modalities. The experimental results on ADNI database demonstrate that the proposed model achieves a prediction accuracy of 86.79% for AD, MCI and NC, which is higher than the existing multi-modal AD prediction models.

engineering, biomedical,computer science, interdisciplinary applications,mathematical & computational biology,biology

WISE 2014 Challenge: Multi-label Classification of Print Media Articles to Topics

Multimodal classification of Alzheimer's disease and mild cognitive impairment

ASMFS: Adaptive-Similarity-based Multi-modality Feature Selection for Classification of Alzheimer's Disease

Multi-modal Self-paced Locality Preserving Learning for Diagnosis of Alzheimer’s Disease

Multi-modal cross-attention network for Alzheimer's disease diagnosis with multi-modality data

Multi-modal latent space inducing ensemble SVM classifier for early dementia diagnosis with neuroimaging data

Alzheimer’s Disease Diagnosis from Multi-Modal Data Via Feature Inductive Learning and Dual Multilevel Graph Neural Network

Alzheimer's disease diagnosis from multi-modal data via feature inductive learning and dual multilevel graph neural network

Multi-modality sparse representation-based classification for Alzheimer's disease and mild cognitive impairment

Recognition of Alzheimer's disease and Mild Cognitive Impairment with multimodal image-derived biomarkers and Multiple Kernel Learning

Multi-modal discriminative dictionary learning for Alzheimer's disease and mild cognitive impairment

Enhanced Multimodal Low-rank Embedding based Feature Selection Model for Multimodal Alzheimer's Disease Diagnosis

Double-attention Assisted Multi-task Learning for the Alzheimer’s Disease Prediction from Mild Cognitive Impairment*

A hybrid multimodal machine learning model for Detecting Alzheimer's disease

A feature-aware multimodal framework with auto-fusion for Alzheimer's disease diagnosis

An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer's Disease Diagnosis

A Multi-Modal Classification Method for Early Diagnosis of Mild Cognitive Impairment and Alzheimer's Disease Using Three Paradigms With Various Task Difficulties

Self-paced Semi-Supervised Feature Selection with Application to Multi-Modal Alzheimer’s Disease Classification

Deep learning and multimodal feature fusion for the aided diagnosis of Alzheimer's disease

Multimodal Identification of Alzheimer's Disease: A Review

Alzheimer's disease prediction algorithm based on de-correlation constraint and multi-modal feature interaction