Abstract:Background: The authors have previously demonstrated highly reliable automated classification of free-text computed tomography (CT) imaging reports using a hybrid system that pairs linguistic (natural language processing) and statistical (machine learning) techniques. Previously performed for identifying the outcome of orbital fracture in unprocessed radiology reports from a clinical data repository, the performance has not been replicated for more complex outcomes. Objectives: To validate automated outcome classification performance of a hybrid natural language processing (NLP) and machine learning system for brain CT imaging reports. The hypothesis was that our system has performance characteristics for identifying pediatric traumatic brain injury (TBI). Methods: This was a secondary analysis of a subset of 2,121 CT reports from the Pediatric Emergency Care Applied Research Network (PECARN) TBI study. For that project, radiologists dictated CT reports as free text, which were then deidentified and scanned as PDF documents. Trained data abstractors manually coded each report for TBI outcome. Text was extracted from the PDF files using optical character recognition. The data set was randomly split evenly for training and testing. Training patient reports were used as input to the Medical Language Extraction and Encoding (MedLEE) NLP tool to create structured output containing standardized medical terms and modifiers for negation, certainty, and temporal status. A random subset stratified by site was analyzed using descriptive quantitative content analysis to confirm identification of TBI findings based on the National Institute of Neurological Disorders and Stroke (NINDS) Common Data Elements project. Findings were coded for presence or absence, weighted by frequency of mentions, and past/future/indication modifiers were filtered. After combining with the manual reference standard, a decision tree classifier was created using data mining tools WEKA 3.7.5 and Salford Predictive Miner 7.0. Performance of the decision tree classifier was evaluated on the test patient reports. Results: The prevalence of TBI in the sampled population was 159 of 2,217 (7.2%). The automated classification for pediatric TBI is comparable to our prior results, with the notable exception of lower positive predictive value. Manual review of misclassified reports, 95.5% of which were false-positives, revealed that a sizable number of false-positive errors were due to differing outcome definitions between NINDS TBI findings and PECARN clinical important TBI findings and report ambiguity not meeting definition criteria. Conclusions: A hybrid NLP and machine learning automated classification system continues to show promise in coding free-text electronic clinical data. For complex outcomes, it can reliably identify negative reports, but manual review of positive reports may be required. As such, it can still streamline data collection for clinical research and performance improvement.

Natural Language Processing Algorithms Outperform ICD Codes in the Development of Fall Injuries Registry

The use of natural language processing in detecting and predicting falls within the healthcare setting: a systematic review

Automated Identification of Fall-related Injuries in Unstructured Clinical Notes

Natural language processing with deep learning for medical adverse event detection from free-text medical narratives: A case study of detecting total hip replacement dislocation

Developing an Inpatient Electronic Medical Record Phenotype for Hospital-Acquired Pressure Injuries: Case Study Using Natural Language Processing Models

Improved accuracy and efficiency of primary care fall risk screening of older adults using a machine learning approach

A hybrid model to identify fall occurrence from electronic health records

The use of natural language processing for the identification of ageing syndromes including sarcopenia, frailty and falls in electronic healthcare records: a systematic review

Use of Deep Learning to Identify Peripheral Arterial Disease Cases From Narrative Clinical Notes

An Integrated Voice Recognition and Natural Language Processing Platform to Automatically Extract Thoracolumbar Injury Classification Score Features From Radiology Reports

Abstract MP15: Validation of Phenotyping Algorithms for Stroke from Electronic Health Records Using Natural Language Processing

Predicting Falls in Long-term Care Facilities: Machine Learning Study.

Improving postsurgical fall detection for older Americans using LLM-driven analysis of clinical narratives

A Novel Approach for Fall Risk Prediction Using the Inertial Sensor Data From the Timed-Up-and-Go Test in a Community Setting

Systematic evaluation of common natural language processing techniques to codify clinical notes

Using machine learning models to predict falls in hospitalised adults

Classification of neurologic outcomes from medical notes using natural language processing

Using natural language processing for automated classification of disease and to identify misclassified ICD codes in cardiac disease

Automated Outcome Classification of Computed Tomography Imaging Reports for Pediatric Traumatic Brain Injury

Incorporating Natural Language Processing to Improve Classification of Axial Spondyloarthritis Using Electronic Health Records.

Abstract P259: Using Natural Language Processing and Machine Learning to Identify Incident Stroke from Electronic Health Records