Abstract:The prediction of mild cognitive impairment (MCI) conversion to Alzheimer's disease (AD) is important for early treatment to prevent or slow the progression of AD. To accurately predict the MCI conversion to stable MCI or progressive MCI, we propose Triformer, a novel transformer-based framework with three specialized transformers to incorporate multi-model data. Triformer uses I) an image transformer to extract multi-view image features from medical scans, II) a clinical transformer to embed and correlate multi-modal clinical data, and III) a modality fusion transformer that produces an accurate prediction based on fusing the outputs from the image and clinical transformers. Triformer is evaluated on the Alzheimer's Disease Neuroimaging Initiative (ANDI)1 and ADNI2 datasets and outperforms previous state-of-the-art single and multi-modal methods.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **Predict whether patients with mild cognitive impairment (MCI) will convert to Alzheimer's disease (AD)**. Specifically, the author proposes a new Transformer - based framework - TriFormer, which is used to accurately predict whether MCI patients will progress from stable MCI (sMCI) to progressive MCI (pMCI), thereby helping with early intervention to prevent or slow down the development of AD. ### Problem Background MCI is a transitional state between normal cognition and Alzheimer's disease. Approximately 10% to 15% of MCI patients will progress to AD each year. Early non - drug treatments and interventions can delay the conversion of MCI to AD, but only on the premise of accurately predicting the conversion risk of MCI patients. For this reason, using multi - modal data (such as cognitive test results, genetic information, MRI and PET images, etc.) can help more accurately predict the conversion of MCI. ### Limitations of Existing Methods - **Convolutional Neural Network (CNN)**: Although widely used in AD classification and MCI conversion prediction, its strong inductive bias on the local receptive field limits its performance on high - dimensional data. - **Existing Transformer Applications**: Although Transformers perform excellently in capturing global dependencies, due to high computational costs, they are rarely used for MCI conversion prediction of 3D medical image data. ### Innovations of TriFormer To overcome the above problems, the author proposes TriFormer, a framework containing three specially designed Transformer modules: 1. **Image Transformer**: Extract image features from multi - view medical scans. 2. **Clinical Transformer**: Embed and correlate multi - modal clinical data. 3. **Modal Fusion Transformer**: Fuse the outputs of the image and clinical Transformers to generate the final prediction result. ### Experimental Verification TriFormer was evaluated on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) 1 and ADNI2 datasets, and outperformed existing unimodal and multimodal methods on multiple metrics, achieving state - of - the - art results. ### Main Contributions 1. Proposed a 2.5D Vision Transformer to efficiently extract multi - view image features. 2. For the first time, used Transformer to embed and correlate different clinical features, improving the accuracy of MCI conversion prediction. 3. Designed a modal fusion Transformer to combine multi - modal features and further improve the prediction performance. Through these innovations, TriFormer significantly improves the accuracy of MCI conversion prediction, providing strong support for early intervention.

TriFormer: A Multi-modal Transformer Framework For Mild Cognitive Impairment Conversion Prediction

A transformer-based unified multimodal framework for Alzheimer's disease assessment

A Transformer-based Multi-features Fusion Model for Prediction of Conversion in Mild Cognitive Impairment

MMTFN: Multi‐modal multi‐scale transformer fusion network for Alzheimer's disease diagnosis

A multimodal cross-transformer-based model to predict mild cognitive impairment using speech, language and vision

Cascaded Multi-Modal Mixing Transformers for Alzheimer's Disease Classification with Incomplete Data

A Transformer Approach for Cognitive Impairment Classification and Prediction

Hybrid Multimodality Fusion with Cross-Domain Knowledge Transfer to Forecast Progression Trajectories in Cognitive Decline

VGG-TSwinformer: Transformer-based deep learning model for early Alzheimer's disease prediction

Multiple Inputs and Mixed Data for Alzheimer's Disease Classification Based on 3D Vision Transformer

Multimodal ensemble model for Alzheimer's disease conversion prediction from Early Mild Cognitive Impairment subjects

Dual attention based fusion network for MCI Conversion Prediction

Double-attention Assisted Multi-task Learning for the Alzheimer’s Disease Prediction from Mild Cognitive Impairment*

Diagnosis of Alzheimer's disease via optimized lightweight convolution-attention and structural MRI

Multi-level fusion network for mild cognitive impairment identification using multi-modal neuroimages

Prediction of Conversion from Mild Cognitive Impairment to Alzheimer's Disease Using MRI and Structural Network Features.

An End-to-end Multimodal 3D CNN Framework with Multi-level Features for the Prediction of Mild Cognitive Impairment

Longformer: Longitudinal Transformer for Alzheimer's Disease Classification with Structural MRIs

Predicting Conversion of Mild Cognitive Impairments to Alzheimer's Disease and Exploring Impact of Neuroimaging

Predicting conversion of mild cognitive impairment to Alzheimer's disease

Ensemble of Convolutional Neural Networks and Multilayer Perceptron for the Diagnosis of Mild Cognitive Impairment and Alzheimer's Disease