Abstract:Background: Sepsis is a heterogeneous syndrome, and enrollment of more homogeneous patients is essential to improve the efficiency of clinical trials. Artificial intelligence (AI) has facilitated the identification of homogeneous subgroups, but how to estimate the uncertainty of the model outputs when applying AI to clinical decision-making remains unknown. Objective: We aimed to design an AI-based model for purposeful patient enrollment, ensuring that a patient with sepsis recruited into a trial would still be persistently ill by the time the proposed therapy could impact patient outcome. We also expected that the model could provide interpretable factors and estimate the uncertainty of the model outputs at a customized confidence level. Methods: In this retrospective study, 9135 patients with sepsis requiring vasopressor treatment within 24 hours after sepsis onset were enrolled from Beth Israel Deaconess Medical Center. This cohort was used for model development, and 10-fold cross-validation with 50 repeats was used for internal validation. In total, 3743 patients with sepsis from the eICU Collaborative Research Database were used as the external validation cohort. All included patients with sepsis were stratified based on disease progression trajectories: rapid death, recovery, and persistent ill. A total of 148 variables were selected for predicting the 3 trajectories. Four machine learning algorithms with 3 different setups were used. We estimated the uncertainty of the model outputs using conformal prediction (CP). The Shapley Additive Explanations method was used to explain the model. Results: The multiclass gradient boosting machine was identified as the best-performing model with good discrimination and calibration performance in both validation cohorts. The mean area under the receiver operating characteristic curve with SD was 0.906 (0.018) for rapid death, 0.843 (0.008) for recovery, and 0.807 (0.010) for persistent ill in the internal validation cohort. In the external validation cohort, the mean area under the receiver operating characteristic curve (SD) was 0.878 (0.003) for rapid death, 0.764 (0.008) for recovery, and 0.696 (0.007) for persistent ill. The maximum norepinephrine equivalence, total urine output, Acute Physiology Score III, mean systolic blood pressure, and the coefficient of variation of oxygen saturation contributed the most. Compared to the model without CP, using the model with CP at a mixed confidence approach reduced overall prediction errors by 27.6% (n=62) and 30.7% (n=412) in the internal and external validation cohorts, respectively, as well as enabled the identification of more potentially persistent ill patients. Conclusions: The implementation of our model has the potential to reduce heterogeneity and enroll more homogeneous patients in sepsis clinical trials. The use of CP for estimating the uncertainty of the model outputs allows for a more comprehensive understanding of the model's reliability and assists in making informed decisions based on the predicted outcomes.

Raising High-Risk Awareness in Hemodynamic Treatment with Reinforcement Learning for Septic Shock Patients

Physiology-Informed Real-Time Mean Arterial Blood Pressure Learning and Prediction for Septic Patients Receiving Norepinephrine

Individualized resuscitation strategy for septic shock formalized by finite mixture modeling and dynamic treatment regimen

Artificial intelligence can use physiological parameters to optimize treatment strategies and predict clinical deterioration of sepsis in ICU

Unifying cardiovascular modelling with deep reinforcement learning for uncertainty aware control of sepsis treatment

Effectiveness of Automated Alerting System Compared to Usual Care for the Management of Sepsis

Deep reinforcement learning extracts the optimal sepsis treatment policy from treatment records

Offline reinforcement learning with uncertainty for treatment strategies in sepsis

Pediatric septic shock estimation using deep learning and electronic medical records

Optimizing Sepsis Treatment Strategies Via a Reinforcement Learning Model

New Strategies to Optimize Hemodynamics for Sepsis-Associated Encephalopathy

Interpretable Machine-Learning Model for Real-Time, Clustered Risk Factor Analysis of Sepsis and Septic Death in Critical Care

Reinforcement Learning with Balanced Clinical Reward for Sepsis Treatment

Initiation Timing Prediction of Fluid De-Escalation for Patients with Sepsis Using Extreme Gradient Boosting Model

Is Deep Reinforcement Learning Ready for Practical Applications in Healthcare? A Sensitivity Analysis of Duel-DDQN for Hemodynamic Management in Sepsis Patients

Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach

Enhancing Patient Selection in Sepsis Clinical Trials Design Through an AI Enrichment Strategy: Algorithm Development and Validation

A Deep Reinforcement Computation Model for Sepsis Treatment

Reducing patient mortality, length of stay and readmissions through machine learning-based sepsis prediction in the emergency department, intensive care unit and hospital floor units

Real-Time Prediction of Sepsis in Critical Trauma Patients: Machine Learning–Based Modeling Study

Development and Validation of Deep-Learning-Based Sepsis and Septic Shock Early Prediction System (DeepSEPS) Using Real-World ICU Data