Abstract:Background: Stroke is a prevalent disease with a significant global impact. Effective assessment of stroke severity is vital for an accurate diagnosis, appropriate treatment, and optimal clinical outcomes. The National Institutes of Health Stroke Scale (NIHSS) is a widely used scale for quantitatively assessing stroke severity. However, the current manual scoring of NIHSS is labor-intensive, time-consuming, and sometimes unreliable. Applying artificial intelligence (AI) techniques to automate the quantitative assessment of stroke on vast amounts of electronic health records (EHRs) has attracted much interest. Objective: This study aims to develop an automatic, quantitative stroke severity assessment framework through automating the entire NIHSS scoring process on Chinese clinical EHRs. Methods: Our approach consists of two major parts: Chinese clinical named entity recognition (CNER) with a domain-adaptive pre-trained large language model (LLM) and automated NIHSS scoring. To build a high-performing CNER model, we first construct a stroke-specific, densely annotated dataset "Chinese Stroke Clinical Records" (CSCR) from EHRs provided by our partner hospital, based on a stroke ontology that defines semantically related entities for stroke assessment. We then pre-train a Chinese clinical LLM coined "CliRoberta" through domain-adaptive transfer learning and construct a deep learning-based CNER model that can accurately extract entities directly from Chinese EHRs. Finally, an automated, end-to-end NIHSS scoring pipeline is proposed by mapping the extracted entities to relevant NIHSS items and values, to quantitatively assess the stroke severity. Results: Results obtained on a benchmark dataset CCKS2019 and our newly created CSCR dataset demonstrate the superior performance of our domain-adaptive pre-trained LLM and the CNER model, compared with the existing benchmark LLMs and CNER models. The high F1 score of 0.990 ensures the reliability of our model in accurately extracting the entities for the subsequent automatic NIHSS scoring. Subsequently, our automated, end-to-end NIHSS scoring approach achieved excellent inter-rater agreement (0.823) and intraclass consistency (0.986) with the ground truth and significantly reduced the processing time from minutes to a few seconds. Conclusion: Our proposed automatic and quantitative framework for assessing stroke severity demonstrates exceptional performance and reliability through directly scoring the NIHSS from diagnostic notes in Chinese clinical EHRs. Moreover, this study also contributes a new clinical dataset, a pre-trained clinical LLM, and an effective deep learning-based CNER model. The deployment of these advanced algorithms can improve the accuracy and efficiency of clinical assessment, and help improve the quality, affordability and productivity of healthcare services.

Managing Literature Reviews Information through Visualization

Diagnosis of cognitive and motor disorders levels in stroke patients through explainable machine learning based on MRI

Automatic assessment of DWI‐ASPECTS for acute ischemic stroke based on deep learning

Simple Words over Rich Imaging: Accurate Brain Disease Classification via Language Model Analysis of Radiological Reports

Automatic Diagnosis and Subtyping of Ischemic Stroke Based on a Multidimensional Deep Learning System

Automatic quantitative stroke severity assessment based on Chinese clinical named entity recognition with domain-adaptive pre-trained large language model

' s response to reviews Title : A fatal case of recurrent amiodarone-induced thyrotoxicosis after percutaneous tracheostomy

Rapid Computer‐Aided Diagnosis of Stroke by Serum Metabolic Fingerprint Based Multi‐Modal Recognition

Boosting Deep Learning for Interpretable Brain MRI Lesion Detection through the Integration of Radiology Report Information

Assessment of Stroke Risk Using MRI-VPD with Automatic Segmentation of Carotid Plaques and Classification of Plaque Properties Based on Deep Learning

Fusion of Diffusion Weighted MRI and Clinical Data for Predicting Functional Outcome after Acute Ischemic Stroke with Deep Contrastive Learning

Ischemic stroke outcome prediction with diversity features from whole brain tissue using deep learning network

Stroke analysis and recognition in functional near-infrared spectroscopy signals using machine learning methods

Improving the diagnosis of acute ischemic stroke on non-contrast CT using deep learning: a multicenter study

Identification of early invisible acute ischemic stroke in non-contrast computed tomography using two-stage deep-learning model

Research on prognostic risk assessment model for acute ischemic stroke based on imaging and multidimensional data

Enhancing Radiologists' Performance in Detecting Cerebral Aneurysms Using a Deep Learning Model: A Multicenter Study

Application of artificial intelligence‐based magnetic resonance imaging in diagnosis of cerebral small vessel disease

Automatic detection of cognitive impairment in patients with white matter hyperintensity and causal analysis of related factors using artificial intelligence of MRI

Application of Machine Learning Techniques for Characterization of Ischemic Stroke with MRI Images: A Review

Magnetic resonance imaging-based deep learning imaging biomarker for predicting functional outcomes after acute ischemic stroke