Abstract:Background: One of the significant changes in intensive care medicine over the past two decades is the acknowledgement that improper mechanical ventilation settings substantially contribute to pulmonary injury in critically ill patients. Artificial intelligence (AI) solutions can be used to optimize mechanical ventilation settings in intensive care units (ICUs) and to improve patient outcomes. Specifically, machine learning algorithms can be trained on large datasets of patient information and mechanical ventilation settings. These algorithms can then predict patient responses to different ventilation strategies and suggest personalized ventilation settings for individual patients. Objective: In this study, we aimed to design and evaluate an AI solution that could tailor an optimal ventilator strategy for each critically ill patient who requires mechanical ventilation. Methods: We proposed a reinforcement learning-based AI solution using observational data from multiple ICUs in the US. The primary outcome was hospital mortality. Secondary outcomes were the proportion of optimal oxygen saturation and the proportion of optimal mean arterial blood pressure. We trained our AI agent to recommend low/medium/high levels of three ventilator settings - positive end-expiratory pressure (PEEP), fraction of inspired oxygen (FiO2) and ideal body weight-adjusted tidal volume - according to patients' health conditions. We defined a policy as a set of rules guiding ventilator setting changes given specific clinical scenarios. Off-policy evaluation metrics were applied to evaluate the AI policy. Results: We studied 21,595 and 5,105 patients' ICU stays from the eICU Collaborative Research (eICU) and Medical Information Mart for Intensive Care IV (MIMIC-IV) databases respectively. Using the learnt AI policy, we estimated the hospital mortality rate (eICU 12.1±3.1%; MIMIC-IV 29.1±0.9%), proportion of optimal oxygen saturation (eICU 58.7±4.7%; MIMIC-IV 49.0±1.0%), and proportion of optimal mean arterial blood pressure (eICU 31.1±4.5%; MIMIC-IV 41.2±1.0%). Based on multiple quantitative and qualitative evaluation metrics, our proposed AI solution outperformed observed clinical practice.We studied 21,595 5105 and 5,105 21595 patients' ' ICU stays from the Medical Information Mart for Intensive Care -IV (MIMIC-IV)(1) and eICU Collaborative Research (eICU) and Medical Information Mart for Intensive Care IV (MIMIC-IV) databases respectively. Observed hospital mortality rates were 18.2% (eICU) and 31.1% (MIMIC-IV). UsingFor the learnt AI policy, we estimated the hospital mortality rate (eICU 14.7±0.7%; MIMIC-IV 29.1±0.9%), proportion of optimal oxygen saturation (eICU 57.8±1.0%; MIMIC-IV 49.0±1.0%), and proportion of optimal mean arterial blood pressure (eICU 34.7 ± 1.0%; MIMIC-IV 41.2±1.0%). Based on multiple quantitative and qualitative evaluation metrics, our proposed AI solution has potential to outperformed observed clinical practice. Conclusions: Our study found that customizing ventilation settings for individual patients led to lower estimated hospital mortality rates compared to actual rates. This highlights the potential effectiveness of using RL methodology to develop AI models that analyze complex clinical data for optimizing treatment parameters. Additionally, our findings suggest the integration of this model into a clinical decision support system for refining ventilation settings, supporting the need for prospective validation trials.

A Preliminary Approach for Learning Relational Policies for the Management of Critically Ill Children

Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care

Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review

Reinforcement learning for intensive care medicine: actionable clinical insights from novel approaches to reward shaping and off-policy model evaluation

Optimal Treatment Strategies for Critical Patients with Deep Reinforcement Learning

Bias in Reinforcement Learning: A Review in Healthcare Applications

Predicting Clinical Deterioration in Hospitals

The use of machine learning and artificial intelligence within pediatric critical care

Using Machine Learning and Natural Language Processing Algorithms to Automate the Evaluation of Clinical Decision Support in Electronic Medical Record Systems

Secur-e-Health Project: Towards Federated Learning for Smart Pediatric Care

Interpretable Predictive Models for Healthcare via Rational Logistic Regression

Effective Learning of Probabilistic Models for Clinical Predictions from Longitudinal Data

Does Reinforcement Learning Improve Outcomes for Critically Ill Patients? A Systematic Review and Level-of-Readiness Assessment

Rule-Based Models for Risk Estimation and Analysis of In-hospital Mortality in Emergency and Critical Care

Optimizing Medical Treatment for Sepsis in Intensive Care: from Reinforcement Learning to Pre-Trial Evaluation

Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data

Automating Construction of Machine Learning Models With Clinical Big Data: Proposal Rationale and Methods

Predictive and Explainable Analysis of Post-operative Acute Kidney Injury in Children undergoing Cardiopulmonary Bypass: An Application of Large Language Models

Clinical Risk Prediction Using Language Models: Benefits And Considerations

Offline Inverse Constrained Reinforcement Learning for Safe-Critical Decision Making in Healthcare

Reinforcement Learning to Optimize Ventilator Settings for Patients on Invasive Mechanical Ventilation: A Retrospective Study