Predicting Opportunities for Improvement in Trauma Care: A Registry-Based Cohort Study

Jonatan Attergrim,Kelvin Szolnoky,Lovisa Strommer,Olof Brattstrom,Gunilla Wihlke,Martin Jacobsson,Martin Gerdin Warnberg
DOI: https://doi.org/10.1101/2023.01.19.23284654
2024-08-20
Abstract:Importance Trauma quality improvement programs relies on peer review of patient cases to identify opportunities for improvement. Current state-of-the-art systems for selecting patient cases for peer review use audit filters that struggle with poor performance. Objective To develop models predicting opportunities for improvement in trauma care and compare their performance to currently used audit filters. Design, Setting and Participants This single-center registry-based cohort study used data from the trauma centre at Karolinska University Hospital in Stockholm, Sweden, between 2013 and 2023. Participants were adult trauma patients included in the local trauma registry. The models predicting opportunities for improvement in trauma care were developed using logistic regression and the eXtreme Gradient Boosting learner (XGBoost) with an add-one-year-in expanding window approach. Performance was measured using the integrated calibration index (ICI), area under the receiver operating curve (AUC), true positive rates (TPR) and false positive rates (FPR). We compared the performance of the models to locally used audit filters. Main outcome measure Opportunities for improvement, defined as preventable events in patient care with adverse outcomes. These opportunities for improvement were identified by the local peer review processes. Results A total of 8,220 patients were included. The mean (SD) age was 45 (21), 5696 patients (69%) were male, and the mean (SD) injury severity score was 12 (13). Opportunities for improvement were identified in 496 (6%) patients. The logistic regression and XGBoost models were well calibrated with ICIs (95% CI) of 0.032 (0.032-0.032) and 0.033 (0.032-0.033). Compared to the audit filters, both the logistic regression and XGBoost models had higher AUCs (95% CI) of 0.72 (0.717-0.723) and 0.75 (0.747-0.753), TPR (95% CI) of 0.885 (0.881-0.888) and 0.904 (0.901-0.907), and lower FPR (95% CI) of 0.636 (0.635-0.638) and 0.599 (0.598-0.6). The audit filters had an AUC (95% CI) of 0.616 (0.614-0.618), a TPR (95% CI) of 0.903 (0.9-0.906), and a FPR (95% CI) of 0.671 (0.67-0.672). Conclusion and Relevance Both the logistic regression and XGBoost models outperformed audit filters in predicting opportunities for improvement among adult trauma patients and can potentially be used to improve systems for selecting patient cases for trauma peer review.
What problem does this paper attempt to address?
This paper aims to address the issue of predicting quality improvement opportunities in trauma care. Specifically, the goal of the study is to develop models to predict improvement opportunities in trauma care and to compare the performance of these models with the currently used audit filters. The researchers hope to improve the accuracy of identifying trauma cases with improvement opportunities through machine learning methods such as logistic regression and XGBoost, thereby reducing the false positive rate and finding a balance between screening burden and sensitivity. This will help enhance the effectiveness of trauma quality improvement programs.