Abstract:Background: Diagnostic errors are an underappreciated cause of preventable mortality in hospitals and pose a risk for severe patient harm and increase hospital length of stay. Objective: This study aims to explore the potential of machine learning and natural language processing techniques in improving diagnostic safety surveillance. We conducted a rigorous evaluation of the feasibility and potential to use electronic health records clinical notes and existing case review data. Methods: Safety Learning System case review data from 1 large health system composed of 10 hospitals in the mid-Atlantic region of the United States from February 2016 to September 2021 were analyzed. The case review outcome included opportunities for improvement including diagnostic opportunities for improvement. To supplement case review data, electronic health record clinical notes were extracted and analyzed. A simple logistic regression model along with 3 forms of logistic regression models (ie, Least Absolute Shrinkage and Selection Operator, Ridge, and Elastic Net) with regularization functions was trained on this data to compare classification performances in classifying patients who experienced diagnostic errors during hospitalization. Further, statistical tests were conducted to find significant differences between female and male patients who experienced diagnostic errors. Results: In total, 126 (7.4%) patients (of 1704) had been identified by case reviewers as having experienced at least 1 diagnostic error. Patients who had experienced diagnostic error were grouped by sex: 59 (7.1%) of the 830 women and 67 (7.7%) of the 874 men. Among the patients who experienced a diagnostic error, female patients were older (median 72, IQR 66-80 vs median 67, IQR 57-76; P=.02), had higher rates of being admitted through general or internal medicine (69.5% vs 47.8%; P=.01), lower rates of cardiovascular-related admitted diagnosis (11.9% vs 28.4%; P=.02), and lower rates of being admitted through neurology department (2.3% vs 13.4%; P=.04). The Ridge model achieved the highest area under the receiver operating characteristic curve (0.885), specificity (0.797), positive predictive value (PPV; 0.24), and F1-score (0.369) in classifying patients who were at higher risk of diagnostic errors among hospitalized patients. Conclusions: Our findings demonstrate that natural language processing can be a potential solution to more effectively identifying and selecting potential diagnostic error cases for review and therefore reducing the case review burden.

Exploring Clinical Lessons Learned by Experienced Hospitalists from Diagnostic Errors and Successes

Methods for and Challenges in Seeking Diagnostic Excellence

Using Patient Experience Surveys to Identify Potential Diagnostic Safety Breakdowns: A Mixed Methods Study

Hospital commitments to address diagnostic errors: An assessment of 95 US hospitals

"What Else Could It Be?" A Scoping Review of Questions for Patients to Ask Throughout the Diagnostic Process

Diagnostic Errors in Hospitalized Adults Who Died or Were Transferred to Intensive Care

The impact of errors on healthcare professionals in the critical care setting

Reflections on Diagnosis and Diagnostic Errors: a Survey of Internal Medicine Resident and Attending Physicians

Characteristics of Disease-Specific and Generic Diagnostic Pitfalls: A Qualitative Study

The place of error in the teaching field of heart surgery

Toward the eradication of medical diagnostic errors

Enabling diagnostic excellence in the real world: Managing complexity, uncertainty and clinical responsibility

Diagnostic error in mental health: a review

Lessons in clinical reasoning - pitfalls, myths, and pearls: a case of persistent dysphagia and patient partnership

Evaluation of a Natural Language Processing Approach to Identify Diagnostic Errors and Analysis of Safety Learning System Case Review Data: Retrospective Cohort Study

Diagnostic decision-making and strategies to improve diagnosis

Protecting Patients by Reducing Diagnostic Error

The Big Three diagnostic errors through reflections of Japanese internists

Exploring Knowledge of Cognitive Disposition to Respond in Clinical Decision-Making among Early Clinical Learners

Prevalence and Causes of Diagnostic Errors in Hospitalized Patients Under Investigation for COVID-19

Recovery from Medical Errors: The Critical Care Nursing Safety Net