Hospital-wide, dynamic, individualized prediction of central line-associated bloodstream infections - development and temporal evaluation of six prediction models

Elena Albu,Shan GAO,Pieter STIJNEN,Frank E. RADEMAKERS,Christel JANSSENS,Veerle COSSEY,Yves DEBAVEYE,Laure Wynants,Ben Van Calster
DOI: https://doi.org/10.1101/2024.11.04.24316689
2024-11-04
Abstract:Background: Central line-associated bloodstream infections (CLABSI) are preventable hospital-acquired infections. Predicting CLABSI helps improve early intervention strategies and enhance patient safety. Aim: To develop and temporally evaluate dynamic prediction models for continuous CLABSI risk monitoring. Methods: Data from hospitalized patients with central catheter(s) admitted to University Hospitals Leuven between 2014 and 2017 were used to develop five dynamic models (a cause-specific landmark supermodel, two random forest models, and two XGBoost models) to predict 7-day CLABSI risk, accounting for competing events (death, discharge, and catheter removal). The models' predictions were then combined using a superlearner model. All models were temporally evaluated on data from the same hospital from 2018 to 2020 using performance metrics for discrimination, calibration, and clinical utility. Findings: Among 61629 catheter episodes in the training set, 1930 (3.1%) resulted in CLABSI, while in the test set of 44544 catheter episodes, 1059 (2.4%) experienced CLABSI. Among individual models, one XGBoost model reached an AUROC of 0.748. Calibration was good for predicted risks up to 5%, while the cause-specific and XGBoost models overestimated higher predicted risks. The superlearner displayed a modest improvement in discrimination (AUROC up to 0.751) and better calibration than the cause-specific and XGBoost models, but worse than the random forest models. The models showed clinical utility to support standard care interventions (at risk thresholds between 0.5-4%), but not to support advanced interventions (at thresholds 15-25%). A deterioration in model performance over time was observed on temporal evaluation. Conclusion: Hospital-wide CLABSI prediction models offer clinical utility, though temporal evaluation revealed dataset shift.
Health Informatics
What problem does this paper attempt to address?