[Short QT syndrome as a cause of sudden heart stop].

O. Anttonen,L. Kokkonen,H. Huikuri

Abstract:

What problem does this paper attempt to address?

Modeling and Optimization of Epidemiological Control Policies Through Reinforcement Learning

Ishir Rao

DOI: https://doi.org/10.59720/22-157

2024-01-26

Abstract:Pandemics involve the high transmission of a disease that impacts global and local health and economic patterns. The impact of a pandemic can be minimized by enforcing certain restrictions on a community. However, while minimizing infection and death rates, these restrictions can also lead to economic crises. Epidemiological models help propose pandemic control strategies based on non-pharmaceutical interventions such as social distancing, curfews, and lockdowns, reducing the economic impact of these restrictions. However, designing manual control strategies while considering disease spread and economic status is non-trivial. Optimal strategies can be designed through multi-objective reinforcement learning (MORL) models, which demonstrate how restrictions can be used to optimize the outcome of a pandemic. In this research, we utilized an epidemiological Susceptible, Exposed, Infected, Recovered, Deceased (SEIRD) model: a compartmental model for virtually simulating a pandemic day by day. We combined the SEIRD model with a deep double recurrent Q-network to train a reinforcement learning agent to enforce the optimal restriction on the SEIRD simulation based on a reward function. We tested two agents with unique reward functions and pandemic goals to obtain two strategies. The first agent placed long lockdowns to reduce the initial spread of the disease, followed by cyclical and shorter lockdowns to mitigate the resurgence of the disease. The second agent provided similar infection rates but an improved economy by implementing a 10-day lockdown and 20-day no-restriction cycle. This use of reinforcement learning and epidemiological modeling allowed for both economic and infection mitigation in multiple pandemic scenarios.

Artificial Intelligence,Populations and Evolution
Multi-Objective Model-based Reinforcement Learning for Infectious Disease Control

Runzhe Wan,Xinyu Zhang,Rui Song

DOI: https://doi.org/10.48550/arXiv.2009.04607

2022-02-27

Abstract:Severe infectious diseases such as the novel coronavirus (COVID-19) pose a huge threat to public health. Stringent control measures, such as school closures and stay-at-home orders, while having significant effects, also bring huge economic losses. In the face of an emerging infectious disease, a crucial question for policymakers is how to make the trade-off and implement the appropriate interventions timely given the huge uncertainty. In this work, we propose a Multi-Objective Model-based Reinforcement Learning framework to facilitate data-driven decision-making and minimize the overall long-term cost. Specifically, at each decision point, a Bayesian epidemiological model is first learned as the environment model, and then the proposed model-based multi-objective planning algorithm is applied to find a set of Pareto-optimal policies. This framework, combined with the prediction bands for each policy, provides a real-time decision support tool for policymakers. The application is demonstrated with the spread of COVID-19 in China.

Machine Learning
Epidemic Control on a Large-Scale-Agent-Based Epidemiology Model using Deep Deterministic Policy Gradient

Gaurav Deshkar,Jayanta Kshirsagar,Harshal Hayatnagarkar,Janani Venugopalan

2023-04-10

Abstract:To mitigate the impact of the pandemic, several measures include lockdowns, rapid vaccination programs, school closures, and economic stimulus. These interventions can have positive or unintended negative consequences. Current research to model and determine an optimal intervention automatically through round-tripping is limited by the simulation objectives, scale (a few thousand individuals), model types that are not suited for intervention studies, and the number of intervention strategies they can explore (discrete vs continuous). We address these challenges using a Deep Deterministic Policy Gradient (DDPG) based policy optimization framework on a large-scale (100,000 individual) epidemiological agent-based simulation where we perform multi-objective optimization. We determine the optimal policy for lockdown and vaccination in a minimalist age-stratified multi-vaccine scenario with a basic simulation for economic activity. With no lockdown and vaccination (mid-age and elderly), results show optimal economy (individuals below the poverty line) with balanced health objectives (infection, and hospitalization). An in-depth simulation is needed to further validate our results and open-source our framework.

Machine Learning,Systems and Control
[Apolipoproteins in the evaluation of familial susceptibility to atherosclerosis].

A. Jirkovska,J. Válek,D. Grafnetter,K. Vondrá,J. Skibová

1984-02-03

Abstract:
Interpretable AI for policy-making in pandemics

Leonardo Lucio Custode,Giovanni Iacca

DOI: https://doi.org/10.1145/3520304.3533959

2022-05-01

Abstract:Since the first wave of the COVID-19 pandemic, governments have applied restrictions in order to slow down its spreading. However, creating such policies is hard, especially because the government needs to trade-off the spreading of the pandemic with the economic losses. For this reason, several works have applied machine learning techniques, often with the help of special-purpose simulators, to generate policies that were more effective than the ones obtained by governments. While the performance of such approaches are promising, they suffer from a fundamental issue: since such approaches are based on black-box machine learning, their real-world applicability is limited, because these policies cannot be analyzed, nor tested, and thus they are not trustable. In this work, we employ a recently developed hybrid approach, which combines reinforcement learning with evolutionary computation, for the generation of interpretable policies for containing the pandemic. These policies, trained on an existing simulator, aim to reduce the spreading of the pandemic while minimizing the economic losses. Our results show that our approach is able to find solutions that are extremely simple, yet very powerful. In fact, our approach has significantly better performance (in simulated scenarios) than both previous work and government policies.

Machine Learning,Artificial Intelligence,Computers and Society
Optimal Policies for a Pandemic: A Stochastic Game Approach and a Deep Learning Algorithm

Yao Xuan,Robert Balkin,Jiequn Han,Ruimeng Hu,Hector D. Ceniceros

DOI: https://doi.org/10.48550/arXiv.2012.06745

2020-12-12

Optimization and Control

Abstract:Game theory has been an effective tool in the control of disease spread and in suggesting optimal policies at both individual and area levels. In this paper, we propose a multi-region SEIR model based on stochastic differential game theory, aiming to formulate optimal regional policies for infectious diseases. Specifically, we enhance the standard epidemic SEIR model by taking into account the social and health policies issued by multiple region planners. This enhancement makes the model more realistic and powerful. However, it also introduces a formidable computational challenge due to the high dimensionality of the solution space brought by the presence of multiple regions. This significant numerical difficulty of the model structure motivates us to generalize the deep fictitious algorithm introduced in [Han and Hu, MSML2020, pp.221--245, PMLR, 2020] and develop an improved algorithm to overcome the curse of dimensionality. We apply the proposed model and algorithm to study the COVID-19 pandemic in three states: New York, New Jersey, and Pennsylvania. The model parameters are estimated from real data posted by the Centers for Disease Control and Prevention (CDC). We are able to show the effects of the lockdown/travel ban policy on the spread of COVID-19 for each state and how their policies affect each other.
A Microscopic Pandemic Simulator for Pandemic Prediction Using Scalable Million-Agent Reinforcement Learning

Zhenggang Tang,Kai Yan,Liting Sun,Wei Zhan,Changliu Liu

DOI: https://doi.org/10.48550/arXiv.2108.06589

IF: 5.414

2021-08-14

Machine Learning

Abstract:Microscopic epidemic models are powerful tools for government policy makers to predict and simulate epidemic outbreaks, which can capture the impact of individual behaviors on the macroscopic phenomenon. However, existing models only consider simple rule-based individual behaviors, limiting their applicability. This paper proposes a deep-reinforcement-learning-powered microscopic model named Microscopic Pandemic Simulator (MPS). By replacing rule-based agents with rational agents whose behaviors are driven to maximize rewards, the MPS provides a better approximation of real world dynamics. To efficiently simulate with massive amounts of agents in MPS, we propose Scalable Million-Agent DQN (SMADQN). The MPS allows us to efficiently evaluate the impact of different government strategies. This paper first calibrates the MPS against real-world data in Allegheny, US, then demonstratively evaluates two government strategies: information disclosure and quarantine. The results validate the effectiveness of the proposed method. As a broad impact, this paper provides novel insights for the application of DRL in large scale agent-based networks such as economic and social networks.
Exploring the Pareto front of multi-objective COVID-19 mitigation policies using reinforcement learning

Mathieu Reymond,Conor F. Hayes,Lander Willem,Roxana Rădulescu,Steven Abrams,Diederik M. Roijers,Enda Howley,Patrick Mannion,Niel Hens,Ann Nowé,Pieter Libin

DOI: https://doi.org/10.1016/j.eswa.2024.123686

IF: 8.5

2024-03-28

Expert Systems with Applications

Abstract:Infectious disease outbreaks can have a disruptive impact on public health and societal processes. As decision-making in the context of epidemic mitigation is multi-dimensional hence complex, reinforcement learning in combination with complex epidemic models provides a methodology to design refined prevention strategies. Current research focuses on optimizing policies with respect to a single objective, such as the pathogen's attack rate. However, as the mitigation of epidemics involves distinct, and possibly conflicting, criteria (i.a., mortality, morbidity, economic cost, well-being), a multi-objective decision approach is warranted to obtain balanced policies. To enhance future decision-making, we propose a deep multi-objective reinforcement learning approach by building upon a state-of-the-art algorithm called Pareto Conditioned Networks (PCN) to obtain a set of solutions for distinct outcomes of the decision problem. We consider different deconfinement strategies after the first Belgian lockdown within the COVID-19 pandemic and aim to minimize both COVID-19 cases (i.e., infections and hospitalizations) and the societal burden induced by the mitigation measures. As such, we connected a multi-objective Markov decision process with a stochastic compartment model designed to approximate the Belgian COVID-19 waves and explore reactive strategies. As these social mitigation measures are implemented in a continuous action space that modulates the contact matrix of the age-structured epidemic model, we extend PCN to this setting. We evaluate the solution set that PCN returns, and observe that it explored the whole range of possible social restrictions, leading to high-quality trade-offs, as it captured the problem dynamics. In this work, we demonstrate that multi-objective reinforcement learning adds value to epidemiological modeling and provides essential insights to balance mitigation policies.

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Sustainable COVID-19 Policy Responses with Urban Mobility Network Epidemic Models

Yanggang Cheng,Shibo He,Cunqi Shao,Chao Li,Jiming Chen

DOI: https://doi.org/10.1109/tcss.2024.3418622

2024-01-01

IEEE Transactions on Computational Social Systems

Abstract:The COVID-19 pandemic has challenged countries worldwide to strike a balance between implementing epidemic control measures and maintaining economic activity. In response, many countries have adopted sustainable, precise, region-specific, and multilevel prevention and control measures. To apply these measures more effectively and purposefully, it is imperative to quantify their impact on the transmission of COVID-19 within urban areas. Here, we propose a dynamic metapopulation susceptible-exposed-infectious-removed (SEIR) model that incorporates the urban mobility network to simulate the spread of COVID-19 in Beijing and investigate the effects of precise intervention measures. Our proposed model accurately fits the real epidemic trajectory, even with the significant changes in human mobility patterns before and after the epidemic. Additionally, it can also serve as a useful policy evaluation tool by simulating the impact of perturbations in mobility networks on epidemic transmission dynamics. Based on this tool, our results demonstrate that point-of-interest capacity limitation measures can significantly reduce the number of infections with only a minor loss of urban mobility. Furthermore, we show that community dynamic management measures can effectively control and mitigate COVID-19 spread while enabling the normal operation of most economic and social activities. By quantifying the impact of precise intervention measures on new infections and mobility losses, our model enables a cost-benefit analysis of these measures, thus informing targeted and sustainable policy responses to COVID-19.
Planning Multiple Epidemic Interventions with Reinforcement Learning

Anh Mai,Nikunj Gupta,Azza Abouzied,Dennis Shasha

2023-06-07

Abstract:Combating an epidemic entails finding a plan that describes when and how to apply different interventions, such as mask-wearing mandates, vaccinations, school or workplace closures. An optimal plan will curb an epidemic with minimal loss of life, disease burden, and economic cost. Finding an optimal plan is an intractable computational problem in realistic settings. Policy-makers, however, would greatly benefit from tools that can efficiently search for plans that minimize disease and economic costs especially when considering multiple possible interventions over a continuous and complex action space given a continuous and equally complex state space. We formulate this problem as a Markov decision process. Our formulation is unique in its ability to represent multiple continuous interventions over any disease model defined by ordinary differential equations. We illustrate how to effectively apply state-of-the-art actor-critic reinforcement learning algorithms (PPO and SAC) to search for plans that minimize overall costs. We empirically evaluate the learning performance of these algorithms and compare their performance to hand-crafted baselines that mimic plans constructed by policy-makers. Our method outperforms baselines. Our work confirms the viability of a computational approach to support policy-makers

Machine Learning
SIR-RL: Reinforcement Learning for Optimized Policy Control during Epidemiological Outbreaks in Emerging Market and Developing Economies

Maeghal Jain,Ziya Uddin,Wubshet Ibrahim

2024-04-30

Abstract:The outbreak of COVID-19 has highlighted the intricate interplay between public health and economic stability on a global scale. This study proposes a novel reinforcement learning framework designed to optimize health and economic outcomes during pandemics. The framework leverages the SIR model, integrating both lockdown measures (via a stringency index) and vaccination strategies to simulate disease dynamics. The stringency index, indicative of the severity of lockdown measures, influences both the spread of the disease and the economic health of a country. Developing nations, which bear a disproportionate economic burden under stringent lockdowns, are the primary focus of our study. By implementing reinforcement learning, we aim to optimize governmental responses and strike a balance between the competing costs associated with public health and economic stability. This approach also enhances transparency in governmental decision-making by establishing a well-defined reward function for the reinforcement learning agent. In essence, this study introduces an innovative and ethical strategy to navigate the challenge of balancing public health and economic stability amidst infectious disease outbreaks.

Machine Learning,Physics and Society,Populations and Evolution
Game and Reference: Policy Combination Synthesis for Epidemic Prevention and Control

Zhiyi Tan,Bingkun Bao

2024-03-16

Abstract:In recent years, epidemic policy-making models are increasingly being used to provide reference for governors on prevention and control policies against catastrophic epidemics such as SARS, H1N1 and COVID-19. Existing studies are currently constrained by two issues: First, previous methods develop policies based on effect evaluation, since few of factors in real-world decision-making can be modeled, the output policies will then easily become extreme. Second, the subjectivity and cognitive limitation of human make the historical policies not always optimal for the training of decision models. To these ends, we present a novel Policy Combination Synthesis (PCS) model for epidemic policy-making. Specially, to prevent extreme decisions, we introduce adversarial learning between the model-made policies and the real policies to force the output policies to be more human-liked. On the other hand, to minimize the impact of sub-optimal historical policies, we employ contrastive learning to let the model draw on experience from the best historical policies under similar scenarios. Both adversarial and contrastive learning are adaptive based on the comprehensive effects of real policies to ensure the model always learns useful information. Extensive experiments on real-world data prove the effectiveness of the proposed model.

Artificial Intelligence
Precise Mobility Intervention for Epidemic Control Using Unobservable Information Via Deep Reinforcement Learning

Tao Feng,Tong Xia,Xiaochen Fan,Huandong Wang,Zefang Zong,Yong Li

DOI: https://doi.org/10.1145/3534678.3539195

2022-01-01

Abstract:To control the outbreak of COVID-19, efficient individual mobility intervention for EPidemic Control (EPC) strategies are of great importance, which cut off the contact among people at epidemic risks and reduce infections by intervening the mobility of individuals. Reinforcement Learning (RL) is powerful for decision making, however, there are two major challenges in developing an RL-based EPC strategy: (1) the unobservable information about asymptomatic infections in the incubation period makes it difficult for RL's decision-making, and (2) the delayed rewards for RL causes the deficiency of RL learning. Since the results of EPC are reflected in both daily infections (including unobservable asymptomatic infections) and long-term cumulative cases of COVID-19, it is quite daunting to design an RL model for precise mobility intervention. In this paper, we propose a Variational hiErarcHICal reinforcement Learning method for Epidemic control via individual-level mobility intervention, namely Vehicle. To tackle the above challenges, Vehicle first exploits an information rebuilding module that consists of a contact-risk bipartite graph neural network and a variational LSTM to restore the unobservable information. The contact-risk bipartite graph neural network estimates the possibility of an individual being an asymptomatic infection and the risk of this individual spreading the epidemic, as the current state of RL. Then, the Variational LSTM further encodes the state sequence to model the latency of epidemic spreading caused by unobservable asymptomatic infections. Finally, a Hierarchical Reinforcement Learning framework is employed to train Vehicle, which contains dual-level agents to solve the delayed reward problem. Extensive experimental results demonstrate that Vehicle can effectively control the spread of the epidemic. Vehicle outperforms the state-of-the-art baseline methods with remarkably high-precision mobility interventions on both symptomatic and asymptomatic infections.
Pandemic policy assessment by artificial intelligence

Sirui Song,Xue Liu,Yong Li,Yang Yu

DOI: https://doi.org/10.1038/s41598-022-17892-8

2022-08-16

Abstract:Mobility-control policy is a controversial nonpharmacological approach to pandemic control due to its restriction on people's liberty and economic impacts. Due to the computational complexity of mobility control, it is challenging to assess or compare alternative policies. Here, we develop a pandemic policy assessment system that employs artificial intelligence (AI) to evaluate and analyze mobility-control policies. The system includes three components: (1) a general simulation framework that models different policies to comparable network-flow control problems; (2) a reinforcement-learning (RL) oracle to explore the upper-bound execution results of policies; and (3) comprehensive protocols for converting the RL results to policy-assessment measures, including execution complexity, effectiveness, cost and benefit, and risk. We applied the system to real-world metropolitan data and evaluated three popular policies: city lockdown, community quarantine, and route management. For each policy, we generated mobility-pandemic trade-off frontiers. The results manifest that the smartest policies, such as route management, have high execution complexity but limited additional gain from mobility retention. In contrast, a moderate-level intelligent policy such as community quarantine has acceptable execution complexity but can effectively suppress infections and largely mitigate mobility interventions. The frontiers also show one or two turning points, reflecting the safe threshold of mobility retention when considering policy-execution errors. In addition, we simulated different policy environments and found inspirations for the current policy debates on the zero-COVID policy, vaccination policy, and relaxing restrictions.
Game and reference: efficient policy making for epidemic prevention and control

Tan Zhiyi,Bao Bingkun

DOI: https://doi.org/10.1007/s00530-024-01546-y

IF: 3.9

2024-11-29

Multimedia Systems

Abstract:Epidemic policy-making, as a special data-mining task, is proposed to predict the proper intensities of certain epidemic prevention and control policies based on the spatial-temporal data related to regional epidemics. Previous studies are currently constrained by two issues: First, existing methods are all strongly supervised by policy effect evaluation, since only a small proportion of factors in real-world policy-making are modeled, policies made by the existing models are then easily become extreme or unreasonable. Second, the subjectivity and the cognitive limitation of humans make historical policies not always optimal for the training of decision models. To this end, we present a novel P olicy C ombination S ynthesis (PCS) model for epidemic policy-making. In particular, to prevent extreme decisions, we introduce adversarial learning between the model-made policies and the real policies to force the output policies to be more human-like. On the other hand, to minimize the impact of sub-optimal historical policies, we employ contrastive learning to let the model draw on experience from the best historical policies under similar scenarios. Both adversarial learning and contrastive learning are adaptive to the comprehensive effects of real policies, therefore ensuring that the model always learns useful information. Extensive experiments on real-world data show that policies made by the proposed model outperform the baseline models on both the epidemic containment effect and the economic impact, thereby proving the effectiveness of our work.

computer science, information systems, theory & methods
A Game-Theoretic Approach for Hierarchical Epidemic Control

Feiran Jia,Aditya Mate,Zun Li,Shahin Jabbari,Mithun Chakraborty,Milind Tambe,Michael Wellman,Yevgeniy Vorobeychik

DOI: https://doi.org/10.48550/arXiv.2102.10646

2022-08-04

Abstract:We design and analyze a multi-level game-theoretic model of hierarchical policy interventions for epidemic control, such as those in response to the COVID-19 pandemic. Our model captures the potentially mismatched priorities among a hierarchy of policy-makers (e.g., federal, state, and local governments) with respect to two cost components that have opposite dependence on the policy strength -- post-intervention infection rates and the socio-economic cost of policy implementation. Additionally, our model includes a crucial third factor in decisions: a cost of non-compliance with the policy-maker immediately above in the hierarchy, such as non-compliance of counties with state-level policies. We propose two novel algorithms for approximating solutions to such games. The first is based on best response dynamics (BRD), and exploits the tree structure of the game. The second combines quadratic integer programming (QIP), which enables us to collapse the two lowest levels of the game, with best response dynamics. Through extensive experiments, we show that our QIP-based approach significantly outperforms the BRD algorithm both in running time and the quality of equilibrium solutions. Finally, we apply the QIP-based algorithm to experiments based on both synthetic and real-world data under various parameter configurations and analyze the resulting (approximate) equilibria to gain insight into the impact of decentralization on overall welfare (measured as the negative sum of costs) as well as emergent properties like free-riding and fairness in cost distribution among policy-makers.

Computer Science and Game Theory,Multiagent Systems
Hierarchical Multi-agent Model for Reinforced Medical Resource Allocation with Imperfect Information

Qianyue Hao,Fengli Xu,Lin Chen,Pan Hui,Yong Li

DOI: https://doi.org/10.1145/3552436

IF: 5

2022-11-09

ACM Transactions on Intelligent Systems and Technology

Abstract:With the advent of the COVID-19 pandemic, the shortage in medical resources became increasingly more evident. Therefore, efficient strategies for medical resource allocation are urgently needed. However, conventional rule-based methods employed by public health experts have limited capability in dealing with the complex and dynamic pandemic-spreading situation. In addition, model-based optimization methods such as dynamic programming (DP) fail to work since we cannot obtain a precise model in real-world situations most of the time. Model-free reinforcement learning (RL) is a powerful tool for decision-making; however, three key challenges exist in solving this problem via RL: (1) complex situations and countless choices for decision-making in the real world; (2) imperfect information due to the latency of pandemic spreading; and (3) limitations on conducting experiments in the real world since we cannot set up pandemic outbreaks arbitrarily. In this article, we propose a hierarchical RL framework with several specially designed components. We design a decomposed action space with a corresponding training algorithm to deal with the countless choices, ensuring efficient and real-time strategies. We design a recurrent neural network–based framework to utilize the imperfect information obtained from the environment. We also design a multi-agent voting method, which modifies the decision-making process considering the randomness during model training and, thus, improves the performance. We build a pandemic-spreading simulator based on real-world data, serving as the experimental platform. We then conduct extensive experiments. The results show that our method outperforms all baselines, which reduces infections and deaths by 14.25% on average without the multi-agent voting method and up to 15.44% with it.

computer science, information systems, artificial intelligence
A simulation-deep reinforcement learning (SiRL) approach for epidemic control optimization

Büyüktahtakın, İ. Esra

DOI: https://doi.org/10.1007/s10479-022-04926-7

IF: 4.82

2022-09-27

Annals of Operations Research

Abstract:In this paper, we address the controversies of epidemic control planning by developing a novel Simulation-Deep Reinforcement Learning (SiRL) model. COVID-19 reminded constituents over the world that government decision-making could change their lives. During the COVID-19 pandemic, governments were concerned with reducing fatalities as the virus spread but at the same time also maintaining a flowing economy. In this paper, we address epidemic decision-making regarding the interventions necessary given of the epidemic based on the purpose of the decision-maker. Further, we intend to compare different vaccination strategies, such as age-based and random vaccination, to shine a light on who should get priority in the vaccination process. To address these issues, we propose a simulation-deep reinforcement learning (DRL) framework. This framework is composed of an agent-based simulation model and a governor DRL agent that can enforce interventions in the agent-based simulation environment. Computational results show that our DRL agent can learn effective strategies and suggest optimal actions given a specific epidemic situation based on a multi-objective reward structure. We compare our DRL agent's decisions to government interventions at different periods of time during the COVID-19 pandemic. Our results suggest that more could have been done to control the epidemic. In addition, if a random vaccination strategy that allows super-spreaders to get vaccinated early were used, infections would have been reduced by 32% at the expense of 4% more deaths. We also show that a behavioral change of fully quarantining 10% of the risky individuals and using a random vaccination strategy leads to a reduction of the death toll by 14% and 27% compared to the age-based vaccination strategy that was implemented and the New Jersey reported data, respectively. We have also demonstrated the flexibility of our approach to be applied to other locations by validating and applying our model to the COVID-19 case in the state of Kansas.

operations research & management science
Modeling, state estimation, and optimal control for the US COVID-19 outbreak

Calvin Tsay,Fernando Lejarza,Mark A. Stadtherr,Michael Baldea

DOI: https://doi.org/10.1038/s41598-020-67459-8

2020-04-18

Abstract:The novel coronavirus SARS-CoV-2 and resulting COVID-19 disease have had an unprecedented spread and continue to cause an increasing number of fatalities worldwide. While vaccines are still under development, social distancing, extensive testing, and quarantining of confirmed infected subjects remain the most effective measures to contain the pandemic. These measures carry a significant socioeconomic cost. In this work, we introduce a novel optimization-based decision-making framework for managing the COVID-19 outbreak in the US. This includes modeling the dynamics of affected populations, estimating the model parameters and hidden states from data, and an optimal control strategy for sequencing social distancing and testing events such that the number of infections is minimized. The analysis of our extensive computational efforts reveals that social distancing and quarantining are most effective when implemented early, with quarantining of confirmed infected subjects having a much higher impact. Further, we find that "on-off" policies alternating between strict social distancing and relaxing such restrictions can be effective at "flattening" the curve while likely minimizing social and economic cost.

Populations and Evolution,Optimization and Control
Impact of agent-based intervention strategies on the COVID-19 pandemic in large-scale dynamic contact networks

Renfei Wang,Yilin Li,Dayu Wu,Yong Zou,Ming Tang,Shuguang Guan,Ying Liu,Zhen Jin,Efim Pelinovsky,Mikhail Kirillin,Elbert Macau

DOI: https://doi.org/10.1016/j.physa.2024.129852

IF: 3.778

2024-05-29

Physica A Statistical Mechanics and its Applications

Abstract:Many intervention strategies, such as patient isolation and contact tracking, had been implemented in different countries to slow down and control the spread because of the enormous threats and losses caused by COVID-19. Since the contact relationships of millions of people within cities change over time, it is difficult to accurately predict the dynamics of large-scale outbreaks and evaluate the impact of the contact tracking and isolation strategy based on individual contact history on epidemic transmission. Here, we propose a non-markov spreading model based on individual contact dynamic network, to simulate the dynamic contact processes in a city with millions of people. In this model, the historical contact population of each infected person can be backtracked and tracked. Our model can accurately describe the COVID-19 epidemic in Wuhan and Hong Kong. We assess the impact of four agent-based epidemic intervention strategies: travel control, contact tracking and isolation, vaccination, and regular nucleic acid testing to all residents on the epidemic evolution and economic losses. We find that for the original SARS-CoV-2 virus, a strict travel control strategy is effective in both suppressing the spread of COVID-19 and minimizing economic losses. For the Omicron variant (BA.2) with stronger infectious capacity, a relatively loose travel control and an appropriate combination of the other three strategies can effectively control the epidemic outbreak while minimize economic losses. This paper provides an efficient framework for assessing the combination of different agent-based strategies by large-scale simulations in the case of unknown historical contact information of large populations, and the studies on different combinations of control strategies can provide theoretical guidance for future prevention and control.

physics, multidisciplinary

[Short QT syndrome as a cause of sudden heart stop].

Modeling and Optimization of Epidemiological Control Policies Through Reinforcement Learning

Multi-Objective Model-based Reinforcement Learning for Infectious Disease Control

Epidemic Control on a Large-Scale-Agent-Based Epidemiology Model using Deep Deterministic Policy Gradient

[Apolipoproteins in the evaluation of familial susceptibility to atherosclerosis].

Interpretable AI for policy-making in pandemics

Optimal Policies for a Pandemic: A Stochastic Game Approach and a Deep Learning Algorithm

A Microscopic Pandemic Simulator for Pandemic Prediction Using Scalable Million-Agent Reinforcement Learning

Exploring the Pareto front of multi-objective COVID-19 mitigation policies using reinforcement learning

Sustainable COVID-19 Policy Responses with Urban Mobility Network Epidemic Models

Planning Multiple Epidemic Interventions with Reinforcement Learning

SIR-RL: Reinforcement Learning for Optimized Policy Control during Epidemiological Outbreaks in Emerging Market and Developing Economies

Game and Reference: Policy Combination Synthesis for Epidemic Prevention and Control

Precise Mobility Intervention for Epidemic Control Using Unobservable Information Via Deep Reinforcement Learning

Pandemic policy assessment by artificial intelligence

Game and reference: efficient policy making for epidemic prevention and control

A Game-Theoretic Approach for Hierarchical Epidemic Control

Hierarchical Multi-agent Model for Reinforced Medical Resource Allocation with Imperfect Information

A simulation-deep reinforcement learning (SiRL) approach for epidemic control optimization

Modeling, state estimation, and optimal control for the US COVID-19 outbreak

Impact of agent-based intervention strategies on the COVID-19 pandemic in large-scale dynamic contact networks