Abstract:A ventilator is a device that mechanically assists in pumping air into the lungs, which is a life-saving supportive therapy in an intensive care unit (ICU). In clinical scenarios, each patient has unique physiological circumstances and specific respiratory diseases, thus requiring individualized ventilator settings. Long-term supervision by experienced clinicians is essential to perform the task of precisely adjusting ventilator parameters and making timely modifications. Moreover, a tiny clinical error can result in severe lung injury, induce multi-system organ dysfunction, and increase mortality. To reduce the workload of clinicians and prevent medical errors, machine learning (ML), or more specifically, reinforcement learning (RL) methods, have been developed to automatically adjust the ventilator’s parameters and select optimal strategies. However, the ventilator settings contain both continuous (e.g., frequency) and discrete parameters (e.g., ventilation mode), making it challenging for conventional RL-based approaches to handle such problems. Meanwhile, it is necessary to develop models with high data efficiency to overcome medical data insufficiency. In this paper, we propose a model-based hybrid soft actor-critic (MHSAC) algorithm that is developed based on the classic soft actor-critic (SAC) and model-based policy optimization (MBPO) framework. This algorithm can learn both continuous and discrete policies according to the current and predictive state of patient’s physiological information with high data efficiency. Results reveal that our proposed model significantly outperforms the baseline models, achieving superior efficiency and high accuracy in the OpenAI Gym simulation environment. Our proposed model is capable of resolving mixed action space problems, enhancing data efficiency, and accelerating convergence, which can generate practical optimal ventilator settings, minimize possible medical errors, and provide clinical decision support.

Hyper-parameter optimization based on soft actor critic and hierarchical mixture regularization

Hyperparameters Adaptation for Restricted Boltzmann Machines Based on Free Energy

Efficient hyperparameters optimization through model-based reinforcement learning with experience exploiting and meta-learning

Efficient Hyper-parameter Optimization with Cubic Regularization.

Hyperparameter Optimization for Machine Learning Models Based on Bayesian Optimization

Hyperparameter Optimization for Multi-Objective Reinforcement Learning

Hyp-RL : Hyperparameter Optimization by Reinforcement Learning

Agent-based Collaborative Random Search for Hyper-parameter Tuning and Global Function Optimization

Efficient Hyper-parameter Optimization for NLP Applications.

Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Space

Combining Automated Optimisation of Hyperparameters and Reward Shape

Hyperparameter optimization: Classics, acceleration, online, multi-objective, and tools

Large Language Model Agent for Hyper-Parameter Optimization

Discrete Simulation Optimization for Tuning Machine Learning Method Hyperparameters

Rethinking Density Ratio Estimation Based Hyper-Parameter Optimization

Hyper-Parameter Optimization: A Review of Algorithms and Applications

HyperQ-Opt: Q-learning for Hyperparameter Optimization

A Model-Based Hybrid Soft Actor-Critic Deep Reinforcement Learning Algorithm for Optimal Ventilator Settings

An Enhanced Proximal Policy Optimization-Based Reinforcement Learning Method with Random Forest for Hyperparameter Optimization

Hyper: Hyperparameter Robust Efficient Exploration in Reinforcement Learning

Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization