Multimodal Neurodegenerative Disease Subtyping Explained by ChatGPT

Diego Machado Reyes,Hanqing Chao,Juergen Hahn,Li Shen,Pingkun Yan

2024-02-01

Abstract:Alzheimer's disease (AD) is the most prevalent neurodegenerative disease; yet its currently available treatments are limited to stopping disease progression. Moreover, effectiveness of these treatments is not guaranteed due to the heterogenetiy of the disease. Therefore, it is essential to be able to identify the disease subtypes at a very early stage. Current data driven approaches are able to classify the subtypes at later stages of AD or related disorders, but struggle when predicting at the asymptomatic or prodromal stage. Moreover, most existing models either lack explainability behind the classification or only use a single modality for the assessment, limiting scope of its analysis. Thus, we propose a multimodal framework that uses early-stage indicators such as imaging, genetics and clinical assessments to classify AD patients into subtypes at early stages. Similarly, we build prompts and use large language models, such as ChatGPT, to interpret the findings of our model. In our framework, we propose a tri-modal co-attention mechanism (Tri-COAT) to explicitly learn the cross-modal feature associations. Our proposed model outperforms baseline models and provides insight into key cross-modal feature associations supported by known biological mechanisms.

Machine Learning,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to identify disease subtypes in the early stage of Alzheimer's disease (AD). Currently available treatment methods are only limited to halting the progress of the disease, and due to the heterogeneity of the disease, the effectiveness of these treatment methods cannot be guaranteed. Therefore, it becomes crucial to be able to identify disease subtypes at a very early stage. Although existing data - driven methods can classify subtypes of AD or related disorders, they mainly focus on the later stages of the disease and have difficulties in predicting the asymptomatic or prodromal stages. In addition, most existing models either lack interpretability behind the classification or only use a single modality for evaluation, which limits the scope of analysis. Therefore, the paper proposes a multimodal framework that utilizes early indicators such as imaging, genetics, and clinical evaluations to subtype AD patients in the early stage and uses large - language models such as ChatGPT to interpret the findings of the model.

Specifically, the paper aims to:
1. **Solve the early - diagnosis conundrum**: By combining multiple early biomarkers (imaging, genetic, and clinical evaluations), improve the ability to identify subtypes in the early stage of the disease.
2. **Enhance the interpretability of the model**: By introducing the multimodal co - attention mechanism (Tri - COAT), explicitly learn cross - modal feature associations, and use large - language models (such as ChatGPT) to interpret the prediction results of the model.
3. **Overcome the limitations of a single modality**: Most existing models only rely on data from a single modality, which limits the comprehensiveness and accuracy of their analysis. The multimodal framework proposed in the paper can integrate information from different modalities and provide a more comprehensive picture of disease - driving factors.

Through the above methods, the paper hopes to more accurately identify different subtypes of Alzheimer's disease in the early stage, thereby providing support for early intervention and personalized treatment.