Abstract:Background: Sepsis is a heterogeneous syndrome, and enrollment of more homogeneous patients is essential to improve the efficiency of clinical trials. Artificial intelligence (AI) has facilitated the identification of homogeneous subgroups, but how to estimate the uncertainty of the model outputs when applying AI to clinical decision-making remains unknown. Objective: We aimed to design an AI-based model for purposeful patient enrollment, ensuring that a patient with sepsis recruited into a trial would still be persistently ill by the time the proposed therapy could impact patient outcome. We also expected that the model could provide interpretable factors and estimate the uncertainty of the model outputs at a customized confidence level. Methods: In this retrospective study, 9135 patients with sepsis requiring vasopressor treatment within 24 hours after sepsis onset were enrolled from Beth Israel Deaconess Medical Center. This cohort was used for model development, and 10-fold cross-validation with 50 repeats was used for internal validation. In total, 3743 patients with sepsis from the eICU Collaborative Research Database were used as the external validation cohort. All included patients with sepsis were stratified based on disease progression trajectories: rapid death, recovery, and persistent ill. A total of 148 variables were selected for predicting the 3 trajectories. Four machine learning algorithms with 3 different setups were used. We estimated the uncertainty of the model outputs using conformal prediction (CP). The Shapley Additive Explanations method was used to explain the model. Results: The multiclass gradient boosting machine was identified as the best-performing model with good discrimination and calibration performance in both validation cohorts. The mean area under the receiver operating characteristic curve with SD was 0.906 (0.018) for rapid death, 0.843 (0.008) for recovery, and 0.807 (0.010) for persistent ill in the internal validation cohort. In the external validation cohort, the mean area under the receiver operating characteristic curve (SD) was 0.878 (0.003) for rapid death, 0.764 (0.008) for recovery, and 0.696 (0.007) for persistent ill. The maximum norepinephrine equivalence, total urine output, Acute Physiology Score III, mean systolic blood pressure, and the coefficient of variation of oxygen saturation contributed the most. Compared to the model without CP, using the model with CP at a mixed confidence approach reduced overall prediction errors by 27.6% (n=62) and 30.7% (n=412) in the internal and external validation cohorts, respectively, as well as enabled the identification of more potentially persistent ill patients. Conclusions: The implementation of our model has the potential to reduce heterogeneity and enroll more homogeneous patients in sepsis clinical trials. The use of CP for estimating the uncertainty of the model outputs allows for a more comprehensive understanding of the model's reliability and assists in making informed decisions based on the predicted outcomes.

A Quantitative Model For Sepsis Stratification

Development and Validation of a Sepsis Mortality Risk Score for Sepsis-3 Patients in Intensive Care Unit

Machine-Learning-Enabled Prognostic Models for Sepsis

An early sepsis prediction model utilizing machine learning and unbalanced data processing in a clinical context

Optimization of sepsis risk assessment for ward patients

Early Prediction of Mortality, Severity, and Length of Stay in the Intensive Care Unit of Sepsis Patients Based on Sepsis 3.0 by Machine Learning Models

A Machine Learning Model for Early Prediction and Detection of Sepsis in Intensive Care Unit Patients

Development and validation of a prediction model for in-hospital mortality in patients with sepsis

Interpretable Machine-Learning Model for Real-Time, Clustered Risk Factor Analysis of Sepsis and Septic Death in Critical Care

Identifying and analyzing sepsis states: A retrospective study on patients with sepsis in ICUs

Development and Validation of an Ensemble Learning Risk Model for Sepsis after Abdominal Surgery

Construction and validation of a clinical prediction model for sepsis using peripheral perfusion index to predict in-hospital and 28-day mortality risk

Prognostic Models for Sepsis Built on Small Datasets

A Novel Risk-Prediction Scoring System for Sepsis among Patients with Acute Pancreatitis: A Retrospective Analysis of a Large Clinical Database

SepsisLab: Early Sepsis Prediction with Uncertainty Quantification and Active Sensing

Early Warning Models Using Machine Learning to Predict Sepsis-Associated Chronic Critical Illness: A Study Based on the Medical Information Mart for Intensive Care Database

Identification of metabolomics-based prognostic prediction models for ICU septic patients

A machine-learning approach for dynamic prediction of sepsis-induced coagulopathy in critically ill patients with sepsis: an integrated analysis of the MIMIC-IV and eICU-CRD databases

Sepsis Prognosis Related Scoring Standards: a Comprehensive Review

Enhancing Patient Selection in Sepsis Clinical Trials Design Through an AI Enrichment Strategy: Algorithm Development and Validation

A Machine-Learning Approach for Dynamic Prediction of Sepsis-Induced Coagulopathy in Critically Ill Patients With Sepsis