Abstract:Background Since no effective therapies exist for Alzheimer's disease (AD), prevention has become more critical through lifestyle status changes and interventions. Analyzing electronic health records (EHRs) of patients with AD can help us better understand lifestyle's effect on AD. However, lifestyle information is typically stored in clinical narratives. Thus, the objective of the study was to compare different natural language processing (NLP) models on classifying the lifestyle statuses (e.g., physical activity and excessive diet) from clinical texts in English. Methods Based on the collected concept unique identifiers (CUIs) associated with the lifestyle status, we extracted all related EHRs for patients with AD from the Clinical Data Repository (CDR) of the University of Minnesota (UMN). We automatically generated labels for the training data by using a rule-based NLP algorithm. We conducted weak supervision for pre-trained Bidirectional Encoder Representations from Transformers (BERT) models and three traditional machine learning models as baseline models on the weakly labeled training corpus. These models include the BERT base model, PubMedBERT (abstracts + full text), PubMedBERT (only abstracts), Unified Medical Language System (UMLS) BERT, Bio BERT, Bio-clinical BERT, logistic regression, support vector machine, and random forest. The rule-based model used for weak supervision was tested on the GSC for comparison. We performed two case studies: physical activity and excessive diet, in order to validate the effectiveness of BERT models in classifying lifestyle status for all models were evaluated and compared on the developed Gold Standard Corpus (GSC) on the two case studies. Results The UMLS BERT model achieved the best performance for classifying status of physical activity, with its precision, recall, and F-1 scores of 0.93, 0.93, and 0.92, respectively. Regarding classifying excessive diet, the Bio-clinical BERT model showed the best performance with precision, recall, and F-1 scores of 0.93, 0.93, and 0.93, respectively. Conclusion The proposed approach leveraging weak supervision could significantly increase the sample size, which is required for training the deep learning models. By comparing with the traditional machine learning models, the study also demonstrates the high performance of BERT models for classifying lifestyle status for Alzheimer's disease in clinical notes.

Robustness and Sensitivity of BERT Models Predicting Alzheimer's Disease from Text

Augmenting BERT Carefully with Underrepresented Linguistic Features

On Robustness and Bias Analysis of BERT-Based Relation Extraction

How Robust is Your Automatic Diagnosis Model?

Evaluating Text Classification Robustness to Part-of-Speech Adversarial Examples

Exploiting Pre-Trained ASR Models for Alzheimer's Disease Recognition Through Spontaneous Speech

Adversarial Training for Improving Model Robustness? Look at Both Prediction and Interpretation

Deep learning models are not robust against noise in clinical text

Comparing Natural Language Processing Techniques for Alzheimer's Dementia Prediction in Spontaneous Speech

Not All Errors Are Equal: Investigation of Speech Recognition Errors in Alzheimer's Disease Detection

Healing Powers of BERT: How Task-Specific Fine-Tuning Recovers Corrupted Language Models

Too Big to Fail: Larger Language Models are Disproportionately Resilient to Induction of Dementia-Related Linguistic Anomalies

Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT

Extracting Lifestyle Factors for Alzheimer's Disease from Clinical Notes Using Deep Learning with Weak Supervision

al-BERT: a semi-supervised denoising technique for disease prediction

Improving the robustness and accuracy of biomedical language models through adversarial training

InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective

Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attack

Evaluating robustness of language models for chief complaint extraction from patient-generated text

On the Lack of Robust Interpretability of Neural Text Classifiers

Classifying the Lifestyle Status for Alzheimer's Disease from Clinical Notes Using Deep Learning with Weak Supervision.