Abstract:To mitigate the impact of the pandemic, several measures include lockdowns, rapid vaccination programs, school closures, and economic stimulus. These interventions can have positive or unintended negative consequences. Current research to model and determine an optimal intervention automatically through round-tripping is limited by the simulation objectives, scale (a few thousand individuals), model types that are not suited for intervention studies, and the number of intervention strategies they can explore (discrete vs continuous). We address these challenges using a Deep Deterministic Policy Gradient (DDPG) based policy optimization framework on a large-scale (100,000 individual) epidemiological agent-based simulation where we perform multi-objective optimization. We determine the optimal policy for lockdown and vaccination in a minimalist age-stratified multi-vaccine scenario with a basic simulation for economic activity. With no lockdown and vaccination (mid-age and elderly), results show optimal economy (individuals below the poverty line) with balanced health objectives (infection, and hospitalization). An in-depth simulation is needed to further validate our results and open-source our framework.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to balance the impacts on public health and economic activities through optimizing intervention measures during the outbreak of large - scale pandemics (such as COVID - 19). Specifically, the researchers utilized the Deep Deterministic Policy Gradient (DDPG) algorithm to conduct multi - objective optimization in a large - scale agent - based epidemiological model with 100,000 individuals to determine the optimal lockdown and vaccination strategies.
### Background of the Paper
- **Epidemic Situation Background**: The COVID - 19 pandemic has led to more than 460 million infections, 6 million deaths, and a 3% decline in the global GDP in 2020.
- **Existing Intervention Measures**: Governments of various countries have adopted a variety of measures to control the epidemic, including lockdowns, rapid vaccination programs, school closures, and economic stimuli. These measures may bring positive or negative consequences.
- **Limitations of Existing Research**:
- Limited simulation targets.
- Small scale (usually only a few thousand individuals).
- The model type is not suitable for intervention research.
- Limited number of explored intervention strategies (discrete vs continuous).
### Research Objectives
- **Optimization Objectives**: In a large - scale (100,000 individuals) agent - based epidemiological model, determine the optimal lockdown and vaccination strategies through multi - objective optimization.
- **Specific Problems**:
- How to maximize economic activities while minimizing infection and hospitalization rates?
- How to allocate vaccine resources among different age groups?
- How to determine the best time and duration of lockdown?
### Methods
- **Models**:
- **SEIR Model**: Extended to 9 compartments, including Susceptible, Exposed, Asymptomatic, Pre - Symptomatic, Infected Mild, Infected Severe, Hospitalized, etc.
- **Economic Model**: Each household has savings, daily income and expenditure, and the health status of family members affects economic activities.
- **Optimization Algorithm**:
- **DDPG**: Used to handle continuous optimization states and intervention actions, such as the start and end times of lockdowns, vaccination times for different age groups, etc.
- **Evaluation Criteria**:
- **Health Indicators**: The number of mildly infected people and the number of hospitalized people.
- **Economic Indicator**: The number of people below the poverty line.
### Experimental Setup
- **Experimental Parameters**:
- Initial infection rates: 15% and 1%.
- Vaccine effectiveness: high - efficiency and low - efficiency.
- Vaccine availability: high - availability and low - availability.
- **Reward Functions**:
- Health reward ($H_{Rew}$): $- 1\times(InfectedMild + Hospitalized)$.
- Economic reward ($E_{Rew}$): $- 1\times(Below\_Poverty\_Line)$.
- Total reward ($T_{Rew}$): $H_{Rew}+\kappa\times E_{Rew}$, where $\kappa$ is the mixing factor.
### Results
- **Results in Different Scenarios**:
- **High Initial Infection Rate**: Whether lockdown is implemented or not, vaccination can significantly reduce infection and hospitalization rates, but has a greater impact on the economy.
- **Low Initial Infection Rate**: Early vaccination can effectively control the epidemic while having a relatively small impact on the economy.
- **Different Vaccine Availability and Effectiveness**: Even if the vaccine effect is low, extensive vaccination can still significantly reduce infection and hospitalization rates.
### Discussion
- **Main Conclusions**:
- Optimization algorithms (such as DDPG) show in policy informatics...