Abstract:Background: Stroke is a prevalent disease with a significant global impact. Effective assessment of stroke severity is vital for an accurate diagnosis, appropriate treatment, and optimal clinical outcomes. The National Institutes of Health Stroke Scale (NIHSS) is a widely used scale for quantitatively assessing stroke severity. However, the current manual scoring of NIHSS is labor-intensive, time-consuming, and sometimes unreliable. Applying artificial intelligence (AI) techniques to automate the quantitative assessment of stroke on vast amounts of electronic health records (EHRs) has attracted much interest. Objective: This study aims to develop an automatic, quantitative stroke severity assessment framework through automating the entire NIHSS scoring process on Chinese clinical EHRs. Methods: Our approach consists of two major parts: Chinese clinical named entity recognition (CNER) with a domain-adaptive pre-trained large language model (LLM) and automated NIHSS scoring. To build a high-performing CNER model, we first construct a stroke-specific, densely annotated dataset "Chinese Stroke Clinical Records" (CSCR) from EHRs provided by our partner hospital, based on a stroke ontology that defines semantically related entities for stroke assessment. We then pre-train a Chinese clinical LLM coined "CliRoberta" through domain-adaptive transfer learning and construct a deep learning-based CNER model that can accurately extract entities directly from Chinese EHRs. Finally, an automated, end-to-end NIHSS scoring pipeline is proposed by mapping the extracted entities to relevant NIHSS items and values, to quantitatively assess the stroke severity. Results: Results obtained on a benchmark dataset CCKS2019 and our newly created CSCR dataset demonstrate the superior performance of our domain-adaptive pre-trained LLM and the CNER model, compared with the existing benchmark LLMs and CNER models. The high F1 score of 0.990 ensures the reliability of our model in accurately extracting the entities for the subsequent automatic NIHSS scoring. Subsequently, our automated, end-to-end NIHSS scoring approach achieved excellent inter-rater agreement (0.823) and intraclass consistency (0.986) with the ground truth and significantly reduced the processing time from minutes to a few seconds. Conclusion: Our proposed automatic and quantitative framework for assessing stroke severity demonstrates exceptional performance and reliability through directly scoring the NIHSS from diagnostic notes in Chinese clinical EHRs. Moreover, this study also contributes a new clinical dataset, a pre-trained clinical LLM, and an effective deep learning-based CNER model. The deployment of these advanced algorithms can improve the accuracy and efficiency of clinical assessment, and help improve the quality, affordability and productivity of healthcare services.

Study of Relationship of NIHSS and TCM Standardized Sheet of Apoplexy Syndromes Diagnosis base on LVQ Neural Networks

Study on Relationship Between NIHSS and TCM-SSASD Based on the BP Neural Network Multiple Models Method

Novel Prehospital Prediction Model of Large Vessel Occlusion Using Artificial Neural Network

PCV74 NATURAL LANGUAGE PROCESSING-ASSISTED RETROSPECTIVE SCORING OF NIH STROKE SCALE IN CHINA

Automatic quantitative stroke severity assessment based on Chinese clinical named entity recognition with domain-adaptive pre-trained large language model

Quantitative Diagnosis of TCM Syndrome Types Based on Adaptive Resonant Neural Network

Deep learning derived automated ASPECTS on non‐contrast CT scans of acute ischemic stroke patients

Study on intelligent syndrome differentiation neural network model of stomachache in traditional Chinese medicine based on the real world

Construction and Evaluation of Neural Network Correlation Model between Syndrome Elements and Physical and Chemical Indexes of Unstable Angina Pectoris Complicated with Anxiety

Study on cognitive impairment characteristic and TCM syndrome factor of hypertension with lacunar infarction cognitive impairment

Design of Computer-Aided Medical Expert System Based on Machine Adaptive Learning

Design and validation of a new scale for prehospital evaluation of stroke and large vessel occlusion

Research of TCM face diagnosis and symptom factor based on NN

Attention Matching Network for few-shot learning in the syndrome differentiation of cerebral stroke

Clinical application of self-rating somatic symptom scale in silent brain infarction patients with complaints of dizziness

Intelligent cholinergic white matter pathways algorithm based on U-net reflects cognitive impairment in patients with silent cerebrovascular disease

Classification of Stroke Disease Assessment Based on Body Surface Electrical Signals at Acupuncture Points

Markov Clustering Analysis-Based Validation for Traditional Chinese Medicine Syndrome Patterns of Poststroke Cognitive Impairment

Association between Cognition Level and Chinese Traditonal Medical Syndrome in Patients with Amnestic Mild Cerebral Infarction

[Exploring Chinese Medicine and Western Medicine Group Modules in Acute Phase of Ischemic Stroke Disease].

Validation of NINDS-CSN neuropsychological battery for vascular cognitive impairment in Chinese stroke patients