A predictive tool for identification of SARS-CoV-2 PCR-negative emergency department patients using routine test results

Abstract

Background: Testing for COVID-19 remains limited in the United States and across the world. Poor allocation of limited testing resources leads to misutilization of health system resources, which complementary rapid testing tools could ameliorate.

Objective: To predict SARS-CoV-2 PCR positivity based on complete blood count components and patient sex.

Study design: A retrospective case-control design for collection of data and a logistic regression prediction model was used. Participants were emergency department patients > 18 years old who had concurrent complete blood counts and SARS-CoV-2 PCR testing. 33 confirmed SARS-CoV-2 PCR positive and 357 negative patients at Stanford Health Care were used for model training. Validation cohorts consisted of emergency department patients > 18 years old who had concurrent complete blood counts and SARS-CoV-2 PCR testing in Northern California (41 PCR positive, 495 PCR negative), Seattle, Washington (40 PCR positive, 306 PCR negative), Chicago, Illinois (245 PCR positive, 1015 PCR negative), and South Korea (9 PCR positive, 236 PCR negative).

Results: A decision support tool that utilizes components of complete blood count and patient sex for prediction of SARS-CoV-2 PCR positivity demonstrated a C-statistic of 78 %, an optimized sensitivity of 93 %, and generalizability to other emergency department populations. By restricting PCR testing to predicted positive patients in a hypothetical scenario of 1000 patients requiring testing but testing resources limited to 60 % of patients, this tool would allow a 33 % increase in properly allocated resources.

Conclusions: A prediction tool based on complete blood count results can better allocate SARS-CoV-2 testing and other health care resources such as personal protective equipment during a pandemic surge.

Keywords: COVID-19; Machine learning; Prediction tool; Rapid testing; SARS-CoV-2.

Fig. 1

Value of a predictive COVID-19 rule-out tool in improving utilization of health care resources during a pandemic. Example contains a cohort comprising 1000 hypothetical patients with respiratory symptoms presenting to emergency departments across a region and assumes 8% COVID-19 prevalence, a highly accurate SARS-CoV-2 test, and 600 of a limited hospital resource (e.g. SARS-CoV-2 tests, personal protective equipment). (Panel A) If patients are randomly tested or randomly allocated a hospital resource during the wait for results, many patients with COVID-19 patients may not get tested or allocated the resource. (Panel B) With availability of a predictive tool of high sensitivity and negative predictive value based on readily available routine test results, utilization of limited confirmatory SARS-CoV-2 testing or other resources is reserved for those patients more likely to have COVID-19, with a 33 % improvement (48/80 to 74/80) in resource allocation. COVID19 +ve: COVID-19 positive patients; COVID19 -ve: COVID-19 negative patients; Pred + ve: predicted positive; Pred -ve: predicted negative.

Fig. 2

Performance of complete blood count (CBC)-based predictive COVID-19 rule-out tool. A) Results of predictive tool on Stanford Health Care emergency department patient cohort. B) Results of predictive tool on a Seattle, Washington emergency department patient cohort. C) Results of predictive tool on a Chicago, Illinois emergency department patient cohort. D) South Korean cohort of emergency department patients. All four cohorts represented validation sets not previously seen by the decision support tool. SARS-CoV-2 PCR performed in local laboratories was used as the reference method. Left: Receiver operating characteristic curves. Middle: Specificity versus negative predictive value across all operating thresholds. Right: Confusion matrix calculated using operating point defined using Stanford Health Care training cohort. AUC: Receiver operating characteristic area under curve. PPV: positive predictive value. NPV: negative predictive value. Avg. NPV: Weighted average of negative predictive values with specificity as weights across all probability thresholds. Always neg. model: baseline negative predictive value expected by a classifier that always predicts SARS-CoV-2 negative.