Abstract:The COVID-19 pandemic emerged in late December 2019. In the first six months of the global outbreak, the U.S. reported more cases and deaths than any other country in the world. Effective modeling of the course of the pandemic can help assist with public health resource planning, intervention efforts, and vaccine clinical trials. However, building applied forecasting models presents unique challenges during a pandemic. First, case data available to models in real time represent a nonstationary fraction of the true case incidence due to changes in available diagnostic tests and test-seeking behavior. Second, interventions varied across time and geography leading to large changes in transmissibility over the course of the pandemic. We propose a mechanistic Bayesian model that builds upon the classic compartmental susceptible-exposed-infected-recovered (SEIR) model to operationalize COVID-19 forecasting in real time. This framework includes nonparametric modeling of varying transmission rates, nonparametric modeling of case and death discrepancies due to testing and reporting issues, and a joint observation likelihood on new case counts and new deaths; it is implemented in a probabilistic programming language to automate the use of Bayesian reasoning for quantifying uncertainty in probabilistic forecasts. The model has been used to submit forecasts to the U.S. Centers for Disease Control through the COVID-19 Forecast Hub under the name MechBayes. We examine the performance relative to a baseline model as well as alternate models submitted to the forecast hub. Additionally, we include an ablation test of our extensions to the classic SEIR model. We demonstrate a significant gain in both point and probabilistic forecast scoring measures using MechBayes, when compared to a baseline model, and show that MechBayes ranks as one of the top two models out of nine which regularly submitted to the COVID-19 Forecast Hub for the duration of the pandemic, trailing only the COVID-19 Forecast Hub ensemble model of which which MechBayes is a part.

Caution Warranted: Using the Institute for Health Metrics and Evaluation Model for Predicting the Course of the COVID-19 Pandemic

Forecasting the Long-Term Trend of COVID-19 Epidemic Using a Dynamic Model

Learning as We Go: An Examination of the Statistical Accuracy of COVID19 Daily Death Count Predictions

A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state

The Use and Misuse of Mathematical Modeling for Infectious Disease Policymaking: Lessons for the COVID-19 Pandemic

Accuracy of US CDC COVID-19 forecasting models

Validity and usefulness of COVID-19 models

Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty

COVID-19 Outbreak in Wuhan Demonstrates the Limitations of Publicly Available Case Numbers for Epidemiological Modeling

Modeling and prediction of COVID-19 in the United States considering population behavior and vaccination

The disutility of compartmental model forecasts during the COVID-19 pandemic

Assessing the utility of COVID-19 case reports as a leading indicator for hospitalization forecasting in the United States

Modeling of Future COVID-19 Cases, Hospitalizations, and Deaths, by Vaccination Rates and Nonpharmaceutical Intervention Scenarios — United States, April–September 2021

A Hierarchical Bayesian Model for Stochastic Spatiotemporal SIR Modeling and Prediction of COVID-19 Cases and Hospitalizations

Enhancing long-term forecasting: Learning from COVID-19 models

Uncertainty quantification for epidemiological forecasts of COVID-19 through combinations of model predictions

Multiple models for outbreak decision support in the face of uncertainty

REAL-TIME MECHANISTIC BAYESIAN FORECASTS OF COVID-19 MORTALITY

Individual-level Modeling of COVID-19 Epidemic Risk

The Impact of Public Safety Measures on the Spread of COVID-19 in the United States Assessed By Causal Model-Based Projections of the Pandemic

An Information-Motivation-Behavioral Skills (IMB) Model of Pandemic Risk and Prevention