Developing a reference standard for pertussis by applying a stratified sampling strategy to electronic medical record data

Shilo H McBurney; Jeffrey C Kwong; Kevin A Brown; Frank Rudzicz; Branson Chen; Elisa Candido; Natasha S Crowcroft

doi:10.1016/j.annepidem.2022.11.002

Developing a reference standard for pertussis by applying a stratified sampling strategy to electronic medical record data

Ann Epidemiol. 2023 Jan:77:53-60. doi: 10.1016/j.annepidem.2022.11.002. Epub 2022 Nov 11.

Authors

Shilo H McBurney¹, Jeffrey C Kwong², Kevin A Brown³, Frank Rudzicz⁴, Branson Chen⁵, Elisa Candido⁵, Natasha S Crowcroft⁶

Affiliations

¹ Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada. Electronic address: shilo.mcburney@mail.utoronto.ca.
² Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada; Public Health Ontario, Toronto, ON, Canada; Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada; ICES, Toronto, ON, Canada; Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada.
³ Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada; Public Health Ontario, Toronto, ON, Canada; ICES, Toronto, ON, Canada.
⁴ Department of Computer Science, University of Toronto, Toronto, ON, Canada; International Centre for Surgical Safety, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, ON, Canada; Vector Institute for Artificial Intelligence, Toronto, ON, Canada.
⁵ ICES, Toronto, ON, Canada.
⁶ Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada; Immunization, Vaccines and Biologicals, World Health Organization, Geneva, Switzerland.

PMID: 36372292
DOI: 10.1016/j.annepidem.2022.11.002

Abstract

Purpose: Pertussis surveillance remains essential in Canada, but ascertainment bias limits the accuracy of surveillance data. Introducing other sources to improve detection has highlighted the importance of validation. However, challenges arise due to low prevalence, and oversampling suspected cases can introduce partial verification bias. The aim of this study was to build a reference standard for pertussis validation studies that provides adequate analytic precision and minimizes bias.

Methods: We used a stratified strategy to sample the reference standard from a primary care electronic medical record cohort. We incorporated abstractor notes into definite, possible, ruled-out, and no mention of pertussis classifications which were based on surveillance case definitions.

Results: We abstracted eight hundred records from the cohort of 404,922. There were 208 (26%) definite and 261 (32.6%) possible prevalent pertussis cases. Classifications demonstrated a wide variety of case severities. Abstraction reliability was moderate to substantial based on Cohen's kappa and raw percent agreement.

Conclusions: When conducting validation studies for pertussis and other low prevalence diseases, this stratified sampling strategy can be used to develop a reference standard using limited resources. This approach mitigates verification and spectrum bias while providing sufficient precision and incorporating a range of case severities.

Keywords: Diagnostic accuracy; Low prevalence; Pertussis; Reference standard; Sampling strategy; Validation.

MeSH terms

Canada / epidemiology
Electronic Health Records*
Humans
Reference Standards
Reproducibility of Results
Whooping Cough* / diagnosis
Whooping Cough* / epidemiology

Grants and funding

001/World Health Organization/International