Abstract:Background: Early detection of cognitive decline during the preclinical stage of Alzheimer's disease is crucial for timely intervention and treatment. Clinical notes, often found in unstructured electronic health records (EHRs), contain valuable information that can aid in the early identification of cognitive decline. In this study, we utilize advanced large clinical language models, fine-tuned on clinical notes, to improve the early detection of cognitive decline. Methods: We collected clinical notes from 2,166 patients spanning the 4 years preceding their initial mild cognitive impairment (MCI) diagnosis from the Enterprise Data Warehouse (EDW) of Mass General Brigham (MGB). To train the model, we developed SCD-Tron, a large clinical language model on 4,949 note sections labeled by experts. For evaluation, the trained model was applied to 1,996 independent note sections to assess its performance on real-world unstructured clinical data. Additionally, we used explainable AI techniques, specifically SHAP values, to interpret the model's predictions and provide insight into the most influential features. Error analysis was also facilitated to further analyze the model's prediction. Results: SCD-Tron significantly outperforms baseline models, achieving notable improvements in precision, recall, and AUC metrics for detecting Subjective Cognitive Decline (SCD). Tested on many real-world clinical notes, SCD-Tron demonstrated high sensitivity with only one false negative, crucial for clinical applications prioritizing early and accurate SCD detection. SHAP-based interpretability analysis highlighted key textual features contributing to model predictions, supporting transparency and clinician understanding. Conclusion: SCD-Tron offers a novel approach to early cognitive decline detection by applying large clinical language models to unstructured EHR data. Pretrained on real-world clinical notes, it accurately identifies early cognitive decline and integrates SHAP for interpretability, enhancing transparency in predictions.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to detect cognitive decline early in the pre - clinical stage of Alzheimer's disease (AD). Specifically, researchers use a large clinical language model to analyze unstructured clinical notes in Electronic Health Records (EHRs) to improve the early identification ability of Subjective Cognitive Decline (SCD). ### Background and Motivation - **The Severity of Alzheimer's Disease**: Alzheimer's disease is the most common form of dementia. It is estimated that by 2060, nearly 14 million people in the United States will be affected by it. Early detection of Alzheimer's disease is crucial for slowing down the progression of symptoms and improving the quality of life of patients and their caregivers. - **Limitations of Existing Methods**: Existing detection methods such as MRI, PET scans, cerebrospinal fluid analysis, and genetic tests are effective, but they are often invasive, expensive, and resource - intensive, and difficult to be applied on a large scale. - **Advantages of Electronic Health Records**: EHRs are non - invasive, cost - effective, and easily accessible. Clinical notes in EHRs can capture subtle cognitive symptoms, which are helpful for early detection and intervention. ### Research Objectives - **Utilizing a Large Clinical Language Model**: Researchers proposed an AI model named SCD - Tron, which is based on a large clinical language model and aims to detect early cognitive decline from unstructured clinical notes in EHRs. - **Enhancing Model Interpretability**: By integrating SHAP (SHapley Additive exPlanations) technology, the transparency of model predictions is improved, enabling clinicians to understand the basis of the model's decisions. ### Main Contributions 1. **Integrating a Large Clinical Language Model**: This is the first study to apply a large clinical language model to early cognitive decline detection using unstructured clinical notes. 2. **Interpretability in a Clinical Context**: Explaining model predictions through SHAP values provides a new method to understand how large clinical language models make decisions, enhancing transparency and providing actionable information. 3. **Application to Real - World Data**: This study applied these advanced techniques to real - world EHR data, demonstrating the practical value of the model in a clinical setting. ### Method Overview - **Data Collection and Processing**: Clinical notes of 2,166 patients were collected from the Enterprise Data Warehouse (EDW) of Mass General Brigham (MGB) over a period of 4 years before their initial diagnosis of mild cognitive impairment (MCI). Clinical notes were divided into multiple parts and annotated by experts. - **Model Training and Evaluation**: The model was trained with 4,949 annotated parts and evaluated on 1,996 independent parts. The model was pre - trained based on GatorTron and outputs predictions of cognitive decline status (SCD or non - SCD) through a binary classification network. - **Interpretability Technology**: Model predictions were explained through SHAP values to identify the text features that have the greatest impact on the prediction results. ### Experimental Results - **Performance Metrics**: SCD - Tron significantly outperforms the baseline model in terms of precision, recall, and AUC. In particular, it performs outstandingly in recall, which is especially important in clinical applications. - **Error Analysis**: The model shows high sensitivity in practical applications, with only one false - negative case, which is crucial for early detection of cognitive decline. ### Discussion and Future Work - **Limitations of the Model**: The study is based on a specific dataset from a single healthcare system, which may limit the applicability of the model to other populations. In addition, the model mainly relies on text data and fails to integrate other valuable modality data, such as imaging or biomarkers. - **Future Directions**: Future work will expand the scope of SCD - Tron to include multi - modal data, such as MRI scans or genetic information, to improve detection accuracy. At the same time, plans are made to validate the model in a broader and more diverse patient population to ensure its robustness and generalization ability.

SCD-Tron: Leveraging Large Clinical Language Model for Early Detection of Cognitive Decline from Electronic Health Records

Enhancing Early Detection of Cognitive Decline in the Elderly: A Comparative Study Utilizing Large Language Models in Clinical Notes

Development and Validation of a Deep Learning Model for Earlier Detection of Cognitive Decline From Clinical Notes in Electronic Health Records

An explainable machine learning model of cognitive decline derived from speech

Automated Classification of Cognitive Decline and Probable Alzheimer's Dementia Across Multiple Speech and Language Domains

AD-BERT: Using pre-trained language model to predict the progression from mild cognitive impairment to Alzheimer's disease

Assisting Schizophrenia Diagnosis Using Clinical Electroencephalography and Interpretable Graph Neural Networks: a Real-World and Cross-Site Study

Prediction of Clinical Scores for Subjective Cognitive Decline and Mild Cognitive Impairment.

Decoding cognitive health using machine learning: A comprehensive evaluation for diagnosis of significant memory concern

Automated Medical Records Review for Mild Cognitive Impairment and Dementia

Using Deep Learning to Identify Patients with Cognitive Impairment in Electronic Health Records

Cognitive Function Assessment and Prediction for Subjective Cognitive Decline and Mild Cognitive Impairment.

Development of a machine learning model to predict mild cognitive impairment using natural language processing in the absence of screening

Fusing deep learning models for subjective cognitive decline recognition via structural and functional MRI data

Machine Learning for Early Detection of Cognitive Decline in Parkinson's Disease Using Multimodal Biomarker and Clinical Data

Machine Learning-Based Prediction Models for Cognitive Decline Progression: A Comparative Study in Multilingual Settings Using Speech Analysis

Improving clinical efficiency in screening for cognitive impairment due to Alzheimer's

Predicting Risk of Alzheimer's Diseases and Related Dementias with AI Foundation Model on Electronic Health Records

Prediction of 7‐year's conversion from subjective cognitive decline to mild cognitive impairment

Evaluating Large Language Models in Extracting Cognitive Exam Dates and Scores