Abstract:Background: Stroke is a prevalent disease with a significant global impact. Effective assessment of stroke severity is vital for an accurate diagnosis, appropriate treatment, and optimal clinical outcomes. The National Institutes of Health Stroke Scale (NIHSS) is a widely used scale for quantitatively assessing stroke severity. However, the current manual scoring of NIHSS is labor-intensive, time-consuming, and sometimes unreliable. Applying artificial intelligence (AI) techniques to automate the quantitative assessment of stroke on vast amounts of electronic health records (EHRs) has attracted much interest. Objective: This study aims to develop an automatic, quantitative stroke severity assessment framework through automating the entire NIHSS scoring process on Chinese clinical EHRs. Methods: Our approach consists of two major parts: Chinese clinical named entity recognition (CNER) with a domain-adaptive pre-trained large language model (LLM) and automated NIHSS scoring. To build a high-performing CNER model, we first construct a stroke-specific, densely annotated dataset "Chinese Stroke Clinical Records" (CSCR) from EHRs provided by our partner hospital, based on a stroke ontology that defines semantically related entities for stroke assessment. We then pre-train a Chinese clinical LLM coined "CliRoberta" through domain-adaptive transfer learning and construct a deep learning-based CNER model that can accurately extract entities directly from Chinese EHRs. Finally, an automated, end-to-end NIHSS scoring pipeline is proposed by mapping the extracted entities to relevant NIHSS items and values, to quantitatively assess the stroke severity. Results: Results obtained on a benchmark dataset CCKS2019 and our newly created CSCR dataset demonstrate the superior performance of our domain-adaptive pre-trained LLM and the CNER model, compared with the existing benchmark LLMs and CNER models. The high F1 score of 0.990 ensures the reliability of our model in accurately extracting the entities for the subsequent automatic NIHSS scoring. Subsequently, our automated, end-to-end NIHSS scoring approach achieved excellent inter-rater agreement (0.823) and intraclass consistency (0.986) with the ground truth and significantly reduced the processing time from minutes to a few seconds. Conclusion: Our proposed automatic and quantitative framework for assessing stroke severity demonstrates exceptional performance and reliability through directly scoring the NIHSS from diagnostic notes in Chinese clinical EHRs. Moreover, this study also contributes a new clinical dataset, a pre-trained clinical LLM, and an effective deep learning-based CNER model. The deployment of these advanced algorithms can improve the accuracy and efficiency of clinical assessment, and help improve the quality, affordability and productivity of healthcare services.

Nice: Neural In-Hospital Cost Estimation From Medical Records

Predicting Hospital Readmission Via Cost-Sensitive Deep Learning

Cost-sensitive Deep Learning for Early Readmission Prediction at A Major Hospital

In-hospital Resource Utilization Prediction from Electronic Medical Records with Deep Learning

Early prediction of diagnostic-related groups and estimation of hospital cost by processing clinical notes

Cost prediction for ischemic heart disease hospitalization: Interpretable feature extraction using network analysis

Nursing notes-aware deep neural network for predicting mortality risk after hospital discharge

Deep neural models for automated multi-task diagnostic scan management—quality enhancement, view classification and report generation

Prediction of Mortality Events of Patients with Acute Heart Failure in Intensive Care Unit Based on Deep Neural Network

Natural language processing of MIMIC-III clinical notes for identifying diagnosis and procedures with neural networks

A multi-granularity convolutional neural network model with temporal information and attention mechanism for efficient diabetes medical cost prediction

Scalable and accurate deep learning with electronic health records

Modeling Patient Visit Using Electronic Medical Records For Cost Profile Estimation

An Approach for Medical Event Detection in Chinese Clinical Notes of Electronic Health Records

NLP-Based Prediction of Medical Specialties at Hospital Admission Using Triage Notes

FineEHR: Refine Clinical Note Representations to Improve Mortality Prediction

Patient expenditure prediction using deep learning framework

Prediction of Hospital Readmission for Heart Disease - A Deep Learning Approach.

Feature Explanations in Recurrent Neural Networks for Predicting Risk of Mortality in Intensive Care Patients

Automatic quantitative stroke severity assessment based on Chinese clinical named entity recognition with domain-adaptive pre-trained large language model