Some reflections on the test-negative design

Ronald Meester,Jan Bonte

DOI: https://doi.org/10.48550/arXiv.2212.11679

2022-12-22

Applications

Abstract:We discuss some philosophical, methodological and practical problems concerning the use of the test-negative design for COVID-19 vaccines. These problems limit the use of this design considerably.

What problem does this paper attempt to address?

Potential Biases in Test-Negative Design Studies of COVID-19 Vaccine Effectiveness Arising from the Inclusion of Asymptomatic Individuals

Edgar Ortiz-Brizuela,Mabel Carabali,Cong Jiang,Joanna Merckx,Denis Talbot,Mireille E Schnitzer

DOI: https://doi.org/10.1093/aje/kwae288

2024-08-19

American Journal of Epidemiology

Abstract:Abstract The test-negative design (TND) is a popular method for evaluating vaccine effectiveness (VE). A "classical" TND study includes symptomatic individuals tested for the disease targeted by the vaccine to estimate VE against symptomatic infection. However, recent applications of the TND have attempted to estimate VE against infection by including all tested individuals, regardless of their symptoms. In this article, we use directed acyclic graphs and simulations to investigate potential biases in TND studies of COVID-19 VE arising from the use of this "alternative" approach, particularly when applied during periods of widespread testing. We show that the inclusion of asymptomatic individuals can potentially lead to collider stratification bias, uncontrolled confounding by health and healthcare-seeking behaviors (HSBs), and differential outcome misclassification. While our focus is on the COVID-19 setting, the issues discussed here may also be relevant in the context of other infectious diseases. This may be particularly true in scenarios where there is either a high baseline prevalence of infection, a strong correlation between HSBs and vaccination, different testing practices for vaccinated and unvaccinated individuals, or settings where both the vaccine under study attenuates symptoms of infection and diagnostic accuracy is modified by the presence of symptoms.

public, environmental & occupational health
Test negative design for vaccine effectiveness estimation in the context of the COVID-19 pandemic: A systematic methodology review

Miceline Mésidor,Yan Liu,Denis Talbot,Danuta M Skowronski,Gaston De Serres,Joanna Merckx,Anita Koushik,Mina Tadrous,Sara Carazo,Cong Jiang,Mireille E Schnitzer,Danuta M. Skowronski,Mireille E. Schnitzer

DOI: https://doi.org/10.1016/j.vaccine.2023.12.013

IF: 4.169

2023-12-11

Vaccine

Abstract:Background: During the height of the global COVID-19 pandemic, the test-negative design (TND) was extensively used in many countries to evaluate COVID-19 vaccine effectiveness (VE). Typically, the TND involves the recruitment of care-seeking individuals who meet a common clinical case definition. All participants are then tested for an infection of interest. Objectives: To review and describe the variation in TND methodology, and disclosure of potential biases, as applied to the evaluation of COVID-19 VE during the early vaccination phase of the pandemic. Methods: We conducted a systematic review by searching four biomedical databases using defined keywords to identify peer-reviewed articles published between January 1, 2020, and January 25, 2022. We included only original articles that employed a TND to estimate VE of COVID-19 vaccines in which cases and controls were evaluated based on SARS-CoV-2 laboratory test results. Results: We identified 96 studies, 35 of which met the defined criteria. Most studies were from North America (16 studies) and targeted the general population (28 studies). Outcome case definitions were based primarily on COVID-19-like symptoms; however, several papers did not consider or specify symptoms. Cases and controls had the same inclusion criteria in only half of the studies. Most studies relied upon administrative or hospital databases assembled for a different (non-evaluation) clinical purpose. Potential unmeasured confounding (20 studies), misclassification of current SARS-CoV-2 infection (16 studies) and selection bias (10 studies) were disclosed as limitations by some studies. Conclusion: We observed potentially meaningful deviations from the validated design in the application of the TND during the COVID-19 pandemic.

immunology,medicine, research & experimental
Sensitivity Analysis for the Test-Negative Design

Soumyabrata Kundu,Peng Ding,Xinran Li,Jingshu Wang

DOI: https://doi.org/10.48550/arXiv.2406.06980

2024-06-11

Abstract:The test-negative design has become popular for evaluating the effectiveness of post-licensure vaccines using observational data. In addition to its logistical convenience on data collection, the design is also believed to control for the differential health-care-seeking behavior between vaccinated and unvaccinated individuals, which is an important while often unmeasured confounder between the vaccination and infection. Hence, the design has been employed routinely to monitor seasonal flu vaccines and more recently to measure the COVID-19 vaccine effectiveness. Despite its popularity, the design has been questioned, in particular about its ability to fully control for the unmeasured confounding. In this paper, we explore deviations from a perfect test-negative design, and propose various sensitivity analysis methods for estimating the effect of vaccination measured by the causal odds ratio on the subpopulation of individuals with good health-care-seeking behavior. We start with point identification of the causal odds ratio under a test-negative design, considering two forms of assumptions on the unmeasured confounder. These assumptions then lead to two approaches for conducting sensitivity analysis, addressing the influence of the unmeasured confounding in different ways. Specifically, one approach investigates partial control for unmeasured confounder in the test-negative design, while the other examines the impact of unmeasured confounder on both vaccination and infection. Furthermore, these approaches can be combined to provide narrower bounds on the true causal odds ratio, and can be further extended to sharpen the bounds by restricting the treatment effect heterogeneity. Finally, we apply the proposed methods to evaluate the effectiveness of COVID-19 vaccines using observational data from test-negative designs.

Methodology
Double Negative Control Inference in Test-Negative Design Studies of Vaccine Effectiveness

Kendrick Qijun Li,Xu Shi,Wang Miao,Eric Tchetgen Tchetgen

DOI: https://doi.org/10.48550/arXiv.2203.12509

2023-03-09

Abstract:The test-negative design (TND) has become a standard approach to evaluate vaccine effectiveness against the risk of acquiring infectious diseases in real-world settings, such as Influenza, Rotavirus, Dengue fever, and more recently COVID-19. In a TND study, individuals who experience symptoms and seek care are recruited and tested for the infectious disease which defines cases and controls. Despite TND's potential to reduce unobserved differences in healthcare seeking behavior (HSB) between vaccinated and unvaccinated subjects, it remains subject to various potential biases. First, residual confounding may remain due to unobserved HSB, occupation as healthcare worker, or previous infection history. Second, because selection into the TND sample is a common consequence of infection and HSB, collider stratification bias may exist when conditioning the analysis on tested samples, which further induces confounding by latent HSB. In this paper, we present a novel approach to identify and estimate vaccine effectiveness in the target population by carefully leveraging a pair of negative control exposure and outcome variables to account for potential hidden bias in TND studies. We illustrate our proposed method with extensive simulations and an application to study COVID-19 vaccine effectiveness using data from the University of Michigan Health System.

Methodology
Caveats Regarding the Test-Negative Design to Study Mpox (monkeypox) Risk Factors and Vaccine Effectiveness

Isaac Núñez

DOI: https://doi.org/10.1093/aje/kwae284

2024-08-14

Abstract:The test-negative design (TND) has been commonly used to study vaccine effectiveness (notably regarding COVID-19 and influenza vaccines) and has been recently proposed as a valid design to study causal risk factors of diseases during an outbreak. In April 2022, mpox (previously monkeypox) led to a worldwide outbreak that resulted in an international public health emergency. The TND could be used to study vaccine effectiveness and risk factors of mpox using epidemiologic databases, and a few studies have already done so. However, several issues prevent such study design from being valid for this end. Problems stem from stigma surrounding mpox, which impacts a person's decision to look for healthcare. This poses a challenge to the "similar healthcare seeking behaviour" assumption that is central for test negative studies. Further limitations include the differential diagnoses of mpox, which have notable differences from mpox that may be easily detected by clinicians or patients but are unlikely to be included in epidemiologic databases or electronic health records. Herein I discuss caveats regarding the use of the TND in the context of the mpox outbreak, as well as potential steps that may allow it to be used effectively.
Test-negative designs with various reasons for testing: statistical bias and solution

Mengxin Yu,Kendrick Qijun Li,Nicholas Jewell,Eric Tchetgen Tchetgen,Dylan Small,Xu Shi,Bingkai Wang

DOI: https://doi.org/10.48550/arXiv.2312.03967

2023-12-07

Methodology

Abstract:Test-negative designs are widely used for post-market evaluation of vaccine effectiveness. Different from classical test-negative designs where only healthcare-seekers with symptoms are included, recent test-negative designs have involved individuals with various reasons for testing, especially in an outbreak setting. While including these data can increase sample size and hence improve precision, concerns have been raised about whether they will introduce bias into the current framework of test-negative designs, thereby demanding a formal statistical examination of this modified design. In this article, using statistical derivations, causal graphs, and numerical simulations, we show that the standard odds ratio estimator may be biased if various reasons for testing are not accounted for. To eliminate this bias, we identify three categories of reasons for testing, including symptoms, disease-unrelated reasons, and case contact tracing, and characterize associated statistical properties and estimands. Based on our characterization, we propose stratified estimators that can incorporate multiple reasons for testing to achieve consistent estimation and improve precision by maximizing the use of data. The performance of our proposed method is demonstrated through simulation studies.
Severity-dependent test-seeking behaviors and test-negative designs: impact on estimated vaccine effectiveness and utility of analytic and design choices

Avnika B Amin,Matt D T Hitchings,Otavio T Ranzani,Jason R Andrews,Derek A T Cummings,Albert I Ko,Julio Croda,Natalie E Dean

DOI: https://doi.org/10.1093/aje/kwae303

2024-08-27

Abstract:Test-negative designs are increasingly used to evaluate vaccine effectiveness because of desirable properties like reduced confounding due to healthcare-seeking behaviors and lower cost compared to other study designs. An individual's decision to seek care often depends on their disease severity, with severe disease more likely to be captured than mild disease. As many vaccines likely attenuate disease severity, this phenomenon generally results in an upward-biased estimate of vaccine effectiveness against symptomatic disease. To address the resulting bias, analytic solutions like adjusting for or matching on severity have been suggested. In this paper, we examine the performance of the test-negative design under different vaccine effects on disease severity and the utility of adjusting or matching on severity. We further consider the implications of studies that focus only on milder disease by restricting recruitment to outpatient settings. Through an analytic framework and simulations accompanied by a real-world example, we demonstrate that, when vaccination attenuates disease severity, the magnitude of bias is influenced by the degree of under-ascertainment of mild disease relative to severe disease. When vaccination does not attenuate disease severity, bias is not present. We further show that analytic fixes negligibly impact bias and that outpatient-only studies frequently produce downward-biased estimates.
The test-negative design with additional population controls: a practical approach to rapidly obtain information on the causes of the SARS-CoV-2 epidemic

Jan P Vandenbroucke,Elizabeth B Brickley,Christina M J E Vandenbroucke-Grauls,Neil Pearce

DOI: https://doi.org/10.1097/EDE.0000000000001251

2020-07-07

Abstract:Testing of symptomatic persons for infection with SARS-CoV-2 is occurring worldwide. We propose two types of case-control studies that can be carried out jointly in test-settings for symptomatic persons. The first, the test-negative case-control design (TND) is the easiest to implement; it only demands collecting information about potential risk factors for COVID-19 from the tested symptomatic persons. The second, standard case-control studies with population controls, requires the collection of data on one or more population controls for each person who is tested in the test facilities, so that test-positives and test-negatives can each be compared with population controls. The TND will detect differences in risk factors between symptomatic persons who have COVID-19 (test-positives) and those who have other respiratory infections (test-negatives). However, risk factors with effect sizes of equal magnitude for both COVID-19 and other respiratory infections will not be identified by the TND. Therefore, we discuss how to add population controls to compare with the test-positives and the test-negatives, yielding two additional case-control studies. We describe two options for population control groups: one composed of accompanying persons to the test facilities, the other drawn from existing country-wide health care databases. We also describe other possibilities for population controls. Combining the TND with population controls yields a triangulation approach that distinguishes between exposures that are risk factors for both COVID-19 and other respiratory infections, and exposures that are risk factors for just COVID-19. This combined design can be applied to future epidemics, but also to study causes of non-epidemic disease.

Populations and Evolution,Applications
Test-negative Design:Basic Principles,Methods of Implementation,Types of Derivatives,Advantages, Limitations,and Applications

WU Ting,LIU Jue

DOI: https://doi.org/10.12114/j.issn.1007-9572.2022.0277

2022-01-01

Chinese General Practice

Abstract:As a simple, rapid and effective research method, test-negative design (TND) has been widely used to support the evaluation of post-marketing effectiveness of vaccines and efficacies of interventions in healthcare institutions, showing a magnificent prospect of application. With the emergence of new derivative types such as real-time TND and cluster-randomized TND, TND has also been gradually applied to the exploration of disease risk factors and effectiveness evaluation of interventions. However, there are still few related research reports in China. We introduced the basic principles, methods and essentials of implementation, newly derivative types such as real-time TND and cluster-randomized TND, advantages and limitations of TND, as well as its applications in assessing post-marketing effectiveness for vaccines and efficacies of interventions, providing a theoretical and practical basis for researchers in China to carry out relevant research.
Is "no test is better than a bad test"? Impact of diagnostic uncertainty in mass testing on the spread of COVID-19

Nicholas Gray,Dominic Calleja,Alexander Wimbush,Enrique Miralles-Dolz,Ander Gray,Marco De Angelis,Elfriede Derrer-Merk,Bright Uchenna Oparaji,Vladimir Stepanov,Louis Clearkin,Scott Ferson

DOI: https://doi.org/10.1371/journal.pone.0240775

IF: 3.7

2020-10-21

PLoS ONE

Abstract:Testing is viewed as a critical aspect of any strategy to tackle epidemics. Much of the dialogue around testing has concentrated on how countries can scale up capacity, but the uncertainty in testing has not received nearly as much attention beyond asking if a test is accurate enough to be used. Even for highly accurate tests, false positives and false negatives will accumulate as mass testing strategies are employed under pressure, and these misdiagnoses could have major implications on the ability of governments to suppress the virus. The present analysis uses a modified SIR model to understand the implication and magnitude of misdiagnosis in the context of ending lockdown measures. The results indicate that increased testing capacity alone will not provide a solution to lockdown measures. The progression of the epidemic and peak infections is shown to depend heavily on test characteristics, test targeting, and prevalence of the infection. Antibody based immunity passports are rejected as a solution to ending lockdown, as they can put the population at risk if poorly targeted. Similarly, mass screening for active viral infection may only be beneficial if it can be sufficiently well targeted, otherwise reliance on this approach for protection of the population can again put them at risk. A well targeted active viral test combined with a slow release rate is a viable strategy for continuous suppression of the virus.
Improved Methods for Vaccine Effectiveness Studies

George N Okoli,Benjamin J Cowling

DOI: https://doi.org/10.1093/infdis/jiae510

2024-10-15

The Journal of Infectious Diseases

Abstract:Evaluating the impact of public health investments in vaccination programs is crucial for ensuring their efficiency and effectiveness. Vaccine effectiveness (VE) studies, such as those using the test-negative design (TND), are commonly used to confirm the impact of vaccines and to guide future improvements. The TND, favored for its simplicity and cost efficiency, mitigates biases common in other epidemiological study designs. However, its validity can be compromised by inconsistent symptom definitions and retrospective data application. Here we summarize recent findings that highlight the need to address correlated vaccination behaviors when estimating VE, that suggest using negative control variables to reduce confounding, and that recommend accounting for prior infection history in VE studies to improve accuracy and reliability. These insights are important for refining VE estimation methods.

immunology,infectious diseases,microbiology
Estimating the impact of COVID-19 self-test availability and modifications in test-strategy on overall test uptake using an experimental vignette study

Colene L. Zomer,Floor Kroese,Jet G. Sanders,Riny Janssen,Marijn de Bruin

DOI: https://doi.org/10.1038/s41598-024-54988-9

IF: 4.6

2024-03-13

Scientific Reports

Abstract:To inform future Dutch COVID-19 testing policies we did an experimental vignette study to investigate whether inclusion of the less reliable lateral flow tests (self-tests) would change test-uptake sufficiently to improve population-level test sensitivity. A representative sample (n = 3,270) participated in a 2-by-2 online experiment to evaluate the effects of test-guidelines including self-testing advice (IV1), and the effects of self-test availability (IV2) on expected test uptake (PCR test, self-test or no test) and sensitivity of the overall test strategy (primary outcome). Across four scenarios, changing test advice did not affect expected testing behaviour. Self-test availability, however, increased the timeliness of testing, the number of people testing, and overall test strategy sensitivity. Based on these findings, we recommend that (national) policy facilitates a supply of self-tests at home, for example through free and pro-active distribution of test-kits during a pandemic. This could substantially enhance the chances of timely detecting and isolating patients.

multidisciplinary sciences
Considerations for diagnostic COVID-19 tests

Olivier Vandenberg,Delphine Martiny,Olivier Rochas,Alex van Belkum,Zisis Kozlakidis

DOI: https://doi.org/10.1038/s41579-020-00461-z

IF: 88.1

2020-10-14

Nature Reviews Microbiology

Abstract:During the early phase of the coronavirus disease 2019 (COVID-19) pandemic, design, development, validation, verification and implementation of diagnostic tests were actively addressed by a large number of diagnostic test manufacturers. Hundreds of molecular tests and immunoassays were rapidly developed, albeit many still await clinical validation and formal approval. In this Review, we summarize the crucial role of diagnostic tests during the first global wave of COVID-19. We explore the technical and implementation problems encountered during this early phase in the pandemic, and try to define future directions for the progressive and better use of (syndromic) diagnostics during a possible resurgence of COVID-19 in future global waves or regional outbreaks. Continuous global improvement in diagnostic test preparedness is essential for more rapid detection of patients, possibly at the point of care, and for optimized prevention and treatment, in both industrialized countries and low-resource settings.

microbiology
Adjustment for Disease Severity in the Test-Negative Study Design

Iuliana Ciocănea-Teodorescu,Martha Nason,Arvid Sjölander,Erin E Gabriel

DOI: https://doi.org/10.1093/aje/kwab066

2021-09-01

Abstract:The test-negative study design is often used to estimate vaccine effectiveness in influenza studies, but it has also been proposed in the context of other infectious diseases, such as cholera, dengue, or Ebola. It was introduced as a variation of the case-control design, in an attempt to reduce confounding bias due to health-care-seeking behavior, and has quickly gained popularity because of its logistic advantages. However, examination of the directed acyclic graphs that describe the test-negative design reveals that without strong assumptions, the estimated odds ratio derived under this sampling mechanism is not collapsible over the selection variable, such that the results obtained for the sampled individuals cannot be generalized to the whole population. In this paper, we show that adjustment for severity of disease can reduce this bias and, under certain assumptions, makes it possible to unbiasedly estimate a causal odds ratio. We support our findings with extensive simulations and discuss them in the context of recently published cholera test-negative studies of the effectiveness of cholera vaccines.
Impact of unequal testing on vaccine effectiveness estimates across two study designs: a simulation study

Korryn Bodner,Linwei Wang,Rafal Kustra,Jeffrey C Kwong,Beate Sander,Hind Sbihi,Michael A Irvine,Sharmistha Mishra

DOI: https://doi.org/10.1101/2024.08.27.24312655

2024-08-28

Abstract:Observational studies are essential for measuring vaccine effectiveness. Recent research has raised concerns about how a relationship between testing and vaccination may affect estimates of vaccine effectiveness against symptomatic infection (symptomatic VE). Using an agent-based network model and SARS-CoV-2 as an example, we investigated how differences in the likelihood of testing by vaccination could influence estimates of symptomatic VE across two common study designs: retrospective cohort and test-negative design. First, we measured the influence of unequal testing on symptomatic VE estimates across study designs and sampling periods. Next, we investigated if the magnitude of bias in VE estimates from unequal testing was shaped by the level of immune escape (vaccine efficacy against susceptibility and against infectiousness) and underlying epidemic potential (probability of transmission). We found that unequal testing led to larger biases in the cohort design than the test-negative design and that biases were largest with lower efficacy against susceptibility. We also found the magnitude of bias was moderated by the sampling period, efficacy against infectiousness, and probability of transmission, with more pronounced moderating effects in the test-negative design. Our study illustrates that VE estimates across study designs require careful interpretation, especially in the presence of epidemic and immunological heterogeneity.
Testing practical relevance of treatment effects

Ripamonti, Enrico

DOI: https://doi.org/10.1007/s00362-024-01549-x

2024-04-18

Statistical Papers

Abstract:Traditionally, common testing problems are formalized in terms of a precise null hypothesis representing an idealized situation such as absence of a certain "treatment effect". However, in most applications the real purpose of the analysis is to assess evidence in favor of a practically relevant effect, rather than simply determining its presence/absence. This discrepancy leads to erroneous inferential conclusions, especially in case of moderate or large sample size. In particular, statistical significance, as commonly evaluated on the basis of a precise hypothesis low p value, bears little or no information on practical significance. This paper presents an innovative approach to the problem of testing the practical relevance of effects. This relies upon the proposal of a general method for modifying standard tests by making them suitable to deal with appropriate interval null hypotheses containing all practically irrelevant effect sizes. In addition, when it is difficult to specify exactly which effect sizes are irrelevant we provide the researcher with a benchmark value. Acceptance/rejection can be established purely by deciding on the (ir)relevance of this value. We illustrate our proposal in the context of many important testing setups, and we apply the proposed methods to two case studies in clinical medicine. First, we consider data on the evaluation of systolic blood pressure in a sample of adult participants at risk for nutritional deficit. Second, we focus on a study of the effects of remdesivir on patients hospitalized with COVID-19.

statistics & probability
Who should be tested in a pandemic? Ethical considerations

Sven Ove Hansson,Gert Helgesson,Niklas Juth

DOI: https://doi.org/10.1186/s12910-021-00640-4

2021-06-22

Abstract:Background: In the initial phase of the Covid-19 pandemic, difficult decisions had to be made on the allocation of testing resources. Similar situations can arise in future pandemics. Therefore, careful consideration of who should be tested is an important part of pandemic preparedness. We focus on four ethical aspects of that problem: how to prioritize scarce testing resources, the regulation of commercial direct-to-consumer test services, testing of unauthorized immigrants, and obligatory testing. Main text: The distribution of scarce resources for testing: We emphasize the use of needs-based criteria, but also acknowledge the importance of choosing a testing strategy that contributes efficiently to stopping the overall spread of the disease. Commercial direct-to-consumer test services: Except in cases of acute scarcity, such services will in practice have to be allowed. We propose that they should be subject to regulation that ensures test quality and adequate information to users. Testing of unauthorized immigrants, their children and other people with unclear legal status: Like everyone else, these individuals may be in need of testing, and it is in society's interest to reach them with testing in order to stop the spread of the disease. A society that offers comprehensive medical services to unauthorized immigrants is in a much better position to reach them in a pandemic than a society that previously excluded them from healthcare. Obligatory testing: While there are often strong reasons for universal testing in residential areas or on workplaces, there are in most cases better ways to achieve testing coverage than to make testing mandatory. Conclusion: In summary, we propose (1) decision-making primarily based on needs-based criteria, (2) strict regulation but not prohibition of direct-to-consumer test services, (3) test services offered to unauthorized immigrants, preferably as part of comprehensive medical services, and (4) broad outreach of testing services whenever possible, but in general not obligatory testing.
A Critical Assessment of Some Recent Work on COVID-19

Jörg Stoye

DOI: https://doi.org/10.48550/arXiv.2005.10237

2020-05-26

Abstract:I tentatively re-analyze data from two well-publicized studies on COVID-19, namely the Charité "viral load in children" and the Bonn "seroprevalence in Heinsberg/Gangelt" study, from information available in the preprints. The studies have the following in common: - They received worldwide attention and arguably had policy impact. - The thrusts of their findings align with the respective lead authors' (different) public stances on appropriate response to COVID-19. - Tentatively, my reading of the Gangelt study neutralizes its thrust, and my reading of the Charité study reverses it. The exercise may aid in placing these studies in the literature. With all caveats that apply to n=2 quickfire analyses based off preprints, one also wonders whether it illustrates inadvertent effects of "researcher degrees of freedom."

Applications,Physics and Society,Populations and Evolution
Identifying and alleviating bias due to differential depletion of susceptible people in post-marketing evaluations of COVID-19 vaccines

Rebecca Kahn,Stephanie J. Schrag,Jennifer R. Verani,Marc Lipsitch

DOI: https://doi.org/10.1101/2021.07.15.21260595

2021-07-19

Abstract:Abstract Recent studies have provided key information about SARS-CoV-2 vaccines’ efficacy and effectiveness (VE). One important question that remains is whether the protection conferred by vaccines wanes over time. However, estimates over time are subject to bias from differential depletion of susceptibles between vaccinated and unvaccinated groups. Here we examine the extent to which biases occur under different scenarios and assess whether serologic testing has the potential to correct this bias. By identifying non-vaccine antibodies, these tests could identify individuals with prior infection. We find in scenarios with high baseline VE, differential depletion of susceptibles creates minimal bias in VE estimates, suggesting that any observed declines are likely not due to spurious waning alone. However, if baseline VE is lower, the bias for leaky vaccines (that reduce individual probability of infection given contact) is larger and should be corrected by excluding individuals with past infection if the mechanism is known to be leaky. Conducting analyses both unadjusted and adjusted for past infection could give lower and upper bounds for the true VE. Studies of VE should therefore enroll individuals regardless of prior infection history but also collect information, ideally through serologic testing, on this critical variable.

Some reflections on the test-negative design

Potential Biases in Test-Negative Design Studies of COVID-19 Vaccine Effectiveness Arising from the Inclusion of Asymptomatic Individuals

Test negative design for vaccine effectiveness estimation in the context of the COVID-19 pandemic: A systematic methodology review

Sensitivity Analysis for the Test-Negative Design

Double Negative Control Inference in Test-Negative Design Studies of Vaccine Effectiveness

Caveats Regarding the Test-Negative Design to Study Mpox (monkeypox) Risk Factors and Vaccine Effectiveness

Test-negative designs with various reasons for testing: statistical bias and solution

Severity-dependent test-seeking behaviors and test-negative designs: impact on estimated vaccine effectiveness and utility of analytic and design choices

The test-negative design with additional population controls: a practical approach to rapidly obtain information on the causes of the SARS-CoV-2 epidemic

Test-negative Design:Basic Principles,Methods of Implementation,Types of Derivatives,Advantages, Limitations,and Applications

Is "no test is better than a bad test"? Impact of diagnostic uncertainty in mass testing on the spread of COVID-19

Improved Methods for Vaccine Effectiveness Studies

Estimating the impact of COVID-19 self-test availability and modifications in test-strategy on overall test uptake using an experimental vignette study

Considerations for diagnostic COVID-19 tests

Adjustment for Disease Severity in the Test-Negative Study Design

Impact of unequal testing on vaccine effectiveness estimates across two study designs: a simulation study

Testing practical relevance of treatment effects

Who should be tested in a pandemic? Ethical considerations

A Critical Assessment of Some Recent Work on COVID-19

Identifying and alleviating bias due to differential depletion of susceptible people in post-marketing evaluations of COVID-19 vaccines