Comparison of the Test-negative Design and Cohort Design With Explicit Target Trial Emulation for Evaluating COVID-19 Vaccine Effectiveness

Epidemiology. 2024 Mar 1;35(2):137-149. doi: 10.1097/EDE.0000000000001709. Epub 2023 Dec 18.


Background: Observational studies are used for estimating vaccine effectiveness under real-world conditions. The practical performance of two common approaches-cohort and test-negative designs-need to be compared for COVID-19 vaccines.

Methods: We compared the cohort and test-negative designs to estimate the effectiveness of the BNT162b2 vaccine against COVID-19 outcomes using nationwide data from the United States Department of Veterans Affairs. Specifically, we (1) explicitly emulated a target trial using follow-up data and evaluated the potential for confounding using negative controls and benchmarking to a randomized trial, (2) performed case-control sampling of the cohort to confirm empirically that the same estimate is obtained, (3) further restricted the sampling to person-days with a test, and (4) implemented additional features of a test-negative design. We also compared their performance in limited datasets.

Results: Estimated BNT162b2 vaccine effectiveness was similar under all four designs. Empirical results suggested limited residual confounding by healthcare-seeking behavior. Analyses in limited datasets showed evidence of residual confounding, with estimates biased downward in the cohort design and upward in the test-negative design.

Conclusion: Vaccine effectiveness estimates under a cohort design with explicit target trial emulation and a test-negative design were similar when using rich information from the VA healthcare system, but diverged in opposite directions when using a limited dataset. In settings like ours with sufficient information on confounders and other key variables, the cohort design with explicit target trial emulation may be preferable as a principled approach that allows estimation of absolute risks and facilitates interpretation of effect estimates.

Publication types

  • Randomized Controlled Trial

MeSH terms

  • BNT162 Vaccine
  • COVID-19 Vaccines / therapeutic use
  • COVID-19* / epidemiology
  • COVID-19* / prevention & control
  • Humans
  • United States / epidemiology
  • Vaccine Efficacy
  • Vaccines*


  • COVID-19 Vaccines
  • BNT162 Vaccine
  • Vaccines