Abstract:Stockpiling and scheduling plans for medical supplies represent essential preventive and control measures in major public health events. In the face of major infectious diseases, such as the novel coronavirus disease (COVID-19), the outbreak trend and variability of disease strains are often unpredictable. Hence, it is necessary to optimally adjust the prevention and control dispatching strategy according to the circumstances and outbreak locations to maintain economic development while ensuring the human health survival, however, many models in this scenario seldom consider the dynamic material prediction and the measurement of multiple costs at the same time. Taking the COVID-19 scenario as a case study, we establish a deep reinforcement learning (DRL)-based medical supplies dispatching (MSD) model for major infectious diseases, considering the volatility of the COVID-19 situation and the discrepancy between medical material demand and supply due to the high infectiousness of the Omicron series strains. The present model has three main components: 1) First, for the dynamic medical material prediction problem in complex infectious disease scenarios, taking the lifted COVID-19 lockdown scenario as an example, the modified susceptible-exposed-infected-recovered (SEIR) model was utilized to analyze the spread of the COVID-19, understand its characteristics, and map out the related medical supplies demand; 2) Second, to break away from the previous premise of only considering supply-demand, this study adds scheduling rules and cost function that weighs health and economic costs. An epidemic dispatching optimization model (Epi_DispatchOptim) was established using the OpenAI Gym toolkit to form an environment structure with virus transmission space, and emergency MSD while considering both human health and economic costs. This architecture interprets the balance between the supply-demand of medical supplies and reflects the importance of MSD in the balanced development of health and economy under the spread of infectious diseases; 3) Finally, the MSD strategy under the balance of health and economic cost is explored in Epi_DispatchOptim using reinforcement learning (RL) and the evolutionary algorithm (EA). Experiments conducted on two datasets indicate that the RL and EA reduce economic as well as health costs compared to the original environmental strategies. The above study illustrates how to use epidemiological models to predict the demand for healthcare supplies as the premise of scheduling models, and use Epi_DispatchOptim to explore the dynamic MSD decisions under mortality and economic equilibrium. In Shanghai, China, the economic cost of the exploration strategy is reduced by 27.36–27.07B compared to static scheduling, and deaths are reduced by 126–150 in 150 day compared to the no-intervention scenario. By integrating knowledge of epidemiology, optimal decision making, and economics, Epi_DispatchOptim further constructs epidemiological models, cost functions, state-action spaces, and other modules to assist public health decision makers in adopting appropriate MSD strategies for major public health event.

Recurrent Neural Reinforcement Learning for Counterfactual Evaluation of Public Health Interventions on the Spread of Covid-19 in the world

A Novel Intervention Recurrent Autoencoder for Real Time Forecasting and Non-Pharmaceutical Intervention Selection to Curb the Spread of Covid-19 in the World

Outbreak of Covid 19 worldwide is on the decline Recurrent Neural Reinforcement Learning and Health Interventions to Curb the Spread of Covid 19 in the world

Forecasting the Long-Term Trend of COVID-19 Epidemic Using a Dynamic Model

Forecasting and evaluating intervention of Covid-19 in the World

District-Coupled Epidemic Control via Deep Reinforcement Learning.

Cooperation or Confrontation? —— Prediction of COVID-19 Pandemic Situation Via Deep Learning

Forecasting and Evaluating Multiple Interventions for COVID-19 Worldwide.

Multi-Objective Model-based Reinforcement Learning for Infectious Disease Control

Hierarchical Reinforcement Learning for Scarce Medical Resource Allocation with Imperfect Information

Deep Reinforcement Learning Based Medical Supplies Dispatching Model for Major Infectious Diseases: Case Study of COVID-19

Reverse Engineering the Reproduction Number: A Framework for Data-Driven Counterfactual Analysis, Strategy Evaluation, and Feedback Control of Epidemics

A dynamic approach to support outbreak management using reinforcement learning and semi-connected SEIQR models

Modeling and Optimization of Epidemiological Control Policies Through Reinforcement Learning

A Recurrent Neural Network and Differential Equation Based Spatiotemporal Infectious Disease Model with Application to COVID-19

Contact Tracing and Epidemic Intervention via Deep Reinforcement Learning

SIR-RL: Reinforcement Learning for Optimized Policy Control during Epidemiological Outbreaks in Emerging Market and Developing Economies

Adaptive Epidemic Forecasting and Community Risk Evaluation of COVID-19

Reinforcement Learning Enhances the Experts

HRL4EC: Hierarchical Reinforcement Learning for Multi-Mode Epidemic Control

Big Data Assimilation to Improve the Predictability of COVID-19