Abstract:Observational studies are essential for measuring vaccine effectiveness. Recent research has raised concerns about how a relationship between testing and vaccination may affect estimates of vaccine effectiveness against symptomatic infection (symptomatic VE). Using an agent-based network model and SARS-CoV-2 as an example, we investigated how differences in the likelihood of testing by vaccination could influence estimates of symptomatic VE across two common study designs: retrospective cohort and test-negative design. First, we measured the influence of unequal testing on symptomatic VE estimates across study designs and sampling periods. Next, we investigated if the magnitude of bias in VE estimates from unequal testing was shaped by the level of immune escape (vaccine efficacy against susceptibility and against infectiousness) and underlying epidemic potential (probability of transmission). We found that unequal testing led to larger biases in the cohort design than the test-negative design and that biases were largest with lower efficacy against susceptibility. We also found the magnitude of bias was moderated by the sampling period, efficacy against infectiousness, and probability of transmission, with more pronounced moderating effects in the test-negative design. Our study illustrates that VE estimates across study designs require careful interpretation, especially in the presence of epidemic and immunological heterogeneity.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **The impact of unequal detection rates on vaccine effectiveness (VE) estimates, especially in two commonly used study designs - retrospective cohort study and test - negative design.** Specifically, the authors focus on: 1. **How unequal detection rates affect symptomatic vaccine effectiveness (symptomatic VE) estimates in different study designs**: By using an agent - based network model to simulate the SARS - CoV - 2 epidemic, the biases generated by the two study designs (retrospective cohort and test - negative design) under different testing scenarios were studied. 2. **How these biases are affected by epidemiological and immunological characteristics**: Including the efficacy of the vaccine against susceptibility and infectivity, the underlying epidemic transmission probability, etc. ### Main problem decomposition 1. **The impact of unequal detection rates on VE estimates**: - The authors explored the biases generated by the two study designs (retrospective cohort and test - negative design) under different testing scenarios (such as equal testing, moderately unequal testing, highly unequal testing). - The study found that in the case of unequal detection, both designs would underestimate the true symptomatic VE, but the cohort design had a larger bias. 2. **The impact of epidemiological and immunological characteristics on biases**: - **Efficacy against susceptibility**: Had the greatest impact on the bias, especially more significant in the cohort design. - **Efficacy against infectiousness**: Also had a certain impact, but more obvious in the test - negative design. - **Probability of transmission**: In the case of low transmission probability, small changes would lead to large changes in bias, especially in the test - negative design. ### Research background and motivation In practical applications, observational studies are used to infer the causal effect of vaccines in preventing infection. Accurate VE estimates are crucial for evaluating the impact of vaccines in the population and can affect public confidence in vaccines. However, testing behavior is affected by multiple factors, such as differences in recommended testing, access to medical resources, similar symptoms caused by other diseases, etc. When these factors also affect vaccination status, they may lead to unequal detection rates, thereby introducing residual confounding or selection bias. ### Conclusion The authors found through simulation studies: - In the case of unequal detection, both the cohort design and the test - negative design would underestimate the true symptomatic VE. - The cohort design has a larger bias, especially when the efficacy of the vaccine against susceptibility is low. - The bias of the test - negative design is more easily affected by the sampling period, the efficacy of the vaccine against infectiousness, and the transmission probability. - These findings emphasize the need to consider the heterogeneity of epidemiological and immunological characteristics when interpreting VE estimates. Therefore, this study provides important guidance for future vaccine effectiveness studies, especially in the case of detection inequality, on how to select an appropriate study design to reduce bias.

Impact of unequal testing on vaccine effectiveness estimates across two study designs: a simulation study

Severity-dependent test-seeking behaviors and test-negative designs: impact on estimated vaccine effectiveness and utility of analytic and design choices

Potential Biases in Test-Negative Design Studies of COVID-19 Vaccine Effectiveness Arising from the Inclusion of Asymptomatic Individuals

Identifying and alleviating bias due to differential depletion of susceptible people in post-marketing evaluations of COVID-19 vaccines

Test-negative designs with various reasons for testing: statistical bias and solution

Estimating Waning of Vaccine Effectiveness: a Simulation Study

Quantifying Efficiency Gains of Innovative Designs of Two-Arm Vaccine Trials for COVID-19 Using an Epidemic Simulation Model

Test negative design for vaccine effectiveness estimation in the context of the COVID-19 pandemic: A systematic methodology review

The impact of inaccurate assumptions about antibody test accuracy on the parametrisation and results of infectious disease models of epidemics

Improved Methods for Vaccine Effectiveness Studies

Is "no test is better than a bad test"? Impact of diagnostic uncertainty in mass testing on the spread of COVID-19

Estimation and Hypothesis Testing of Strain-Specific Vaccine Efficacy with Missing Strain Types, with Applications to a COVID-19 Vaccine Trial

Sensitivity Analysis for the Test-Negative Design

Group Testing for SARS-CoV-2 Allows for Up to 10-Fold Efficiency Increase Across Realistic Scenarios and Testing Strategies

Bias in control selection associated with the use of rapid tests in influenza vaccine effectiveness studies

Testing and Isolation Efficacy: Insights from a Simple Epidemic Model

Cost-effective proactive testing strategies during COVID-19 mass vaccination: A modelling study

Simulation and inference on purely observational methods of monitoring vaccine effectiveness post-deployment: none is reliable without precise information on population behaviour

Double Negative Control Inference in Test-Negative Design Studies of Vaccine Effectiveness

How could a pooled testing policy have performed in managing the early stages of the COVID‐19 pandemic? Results from a simulation study