Abstract:Purpose The existing sepsis treatment lacks effective reference and relies too much on the experience of clinicians. Therefore, we used the reinforcement learning model to build an assisted model for the sepsis medication treatment. Methods Using the latest Sepsis 3.0 diagnostic criteria, 19,582 sepsis patients were screened from the Medical Intensive Care Information III database for treatment strategy research, and forty-six features were used in modeling. The study object of the medication strategy is the dosage of vasopressor drugs and intravenous infusion. Dueling DDQN is proposed to predict the patient’s medication strategy (vasopressor and intravenous infusion dosage) through the relationship between the patient’s state, reward function, and medication action. We also constructed protection against the possible high-risk behaviors of Dueling DDQN, especially sudden dose changes of vasopressors can lead to harmful clinical effects. In order to improve the guiding effect of clinically effective medication strategies on the model, we proposed a hybrid model (safe-dueling DDQN + expert strategies) to optimize medication strategies. Results The Dueling DDQN medication model for sepsis patients is superior to clinical strategies and other models in terms of off-policy evaluation values and mortality, and reduced the mortality of clinical strategies from 16.8 to 13.8%. Safe-Dueling DDQN we proposed, compared with Dueling DDQN, has an overall reduction in actions involving vasopressors and reduces large dose fluctuations. The hybrid model we proposed can switch between expert strategies and safe dueling DDQN strategies based on the current state of patients. Conclusions The reinforcement learning model we proposed for sepsis medication treatment, has practical clinical value and can improve the survival rate of patients to a certain extent while ensuring the balance and safety of medication.

Reinforcement Learning For Sepsis Treatment: A Continuous Action Space Solution

Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach

Learning Optimal Treatment Strategies for Sepsis Using Offline Reinforcement Learning in Continuous Space

Model-Based Reinforcement Learning for Sepsis Treatment

Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation

Reinforcement Learning with Balanced Clinical Reward for Sepsis Treatment

Offline reinforcement learning with uncertainty for treatment strategies in sepsis

Dynamic Programming for Solving a Simulated Clinical Scenario of Sepsis Resuscitation

Is Deep Reinforcement Learning Ready for Practical Applications in Healthcare? A Sensitivity Analysis of Duel-DDQN for Hemodynamic Management in Sepsis Patients

Towards more efficient and robust evaluation of sepsis treatment with deep reinforcement learning

Optimal Treatment Strategies for Critical Patients with Deep Reinforcement Learning

Unifying cardiovascular modelling with deep reinforcement learning for uncertainty aware control of sepsis treatment

Deep reinforcement learning extracts the optimal sepsis treatment policy from treatment records

Intelligent Medical Decision Making for Sepsis Detection using Reinforcement Learning

Personalized Vital Signs Control Based on Continuous Action-Space Reinforcement Learning with Supervised Experience.

Optimizing Sepsis Treatment Strategies Via a Reinforcement Learning Model

Reinforcement Learning For Survival, A Clinically Motivated Method For Critically Ill Patients

Clinical knowledge-guided deep reinforcement learning for sepsis antibiotic dosing recommendations

How Consistent are Clinicians? Evaluating the Predictability of Sepsis Disease Progression with Dynamics Models

Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applications

Challenges for Reinforcement Learning in Healthcare