Effect estimates of COVID-19 non-pharmaceutical interventions are non-robust and highly model-dependent

J Clin Epidemiol. 2021 Aug;136:96-132. doi: 10.1016/j.jclinepi.2021.03.014. Epub 2021 Mar 26.


Objective: To compare the inference regarding the effectiveness of the various non-pharmaceutical interventions (NPIs) for COVID-19 obtained from different SIR models.

Study design and setting: We explored two models developed by Imperial College that considered only NPIs without accounting for mobility (model 1) or only mobility (model 2), and a model accounting for the combination of mobility and NPIs (model 3). Imperial College applied models 1 and 2 to 11 European countries and to the USA, respectively. We applied these models to 14 European countries (original 11 plus another 3), over two different time horizons.

Results: While model 1 found that lockdown was the most effective measure in the original 11 countries, model 2 showed that lockdown had little or no benefit as it was typically introduced at a point when the time-varying reproduction number was already very low. Model 3 found that the simple banning of public events was beneficial, while lockdown had no consistent impact. Based on Bayesian metrics, model 2 was better supported by the data than either model 1 or model 3 for both time horizons.

Conclusion: Inferences on effects of NPIs are non-robust and highly sensitive to model specification. In the SIR modeling framework, the impacts of lockdown are uncertain and highly model-dependent.

Keywords: Bayesian statistics; COVID-19; Information criteria; Model comparison; Non-pharmaceutical interventions; SIR models.

MeSH terms

  • COVID-19 / prevention & control*
  • Communicable Disease Control / methods*
  • Europe
  • Humans
  • Models, Statistical*
  • Physical Distancing*
  • Quarantine / methods*
  • SARS-CoV-2