Evaluation of Electronic Medical Record Vital Sign Data Versus a Commercially Available Acuity Score in Predicting Need for Critical Intervention at a Tertiary Children's Hospital

Abstract

Objectives: Evaluate the ability of vital sign data versus a commercially available acuity score adapted for children (pediatric Rothman Index) to predict need for critical intervention in hospitalized pediatric patients to form the foundation for an automated early warning system.

Design: Retrospective review of electronic medical record data.

Setting: Academic children's hospital.

Patients: A total of 220 hospitalized children 6.7 ± 6.7 years old experiencing a cardiopulmonary arrest (condition A) and/or requiring urgent intervention with transfer (condition C) to the ICU between January 2006 and July 2011.

Interventions: None.

Measurements and main results: Physiologic data 24 hours preceding the event were extracted from the electronic medical record. Vital sign predictors were constructed using combinations of age-adjusted abnormalities in heart rate, systolic and diastolic blood pressures, respiratory rate, and peripheral oxygen saturation to predict impending deterioration. Sensitivity and specificity were determined for vital sign-based predictors by using 1:1 age-matched and sex-matched non-ICU control patients. Sensitivity and specificity for a model consisting of any two vital sign measurements simultaneously outside of age-adjusted normal ranges for condition A, condition C, and condition A or C were 64% and 54%, 57% and 53%, and 59% and 54%, respectively. The pediatric Rothman Index (added to the electronic medical record in April 2009) was evaluated in a subset of these patients (n = 131) and 16,138 hospitalized unmatched non-ICU control patients for the ability to predict condition A or C, and receiver operating characteristic curves were generated. Sensitivity and specificity for a pediatric Rothman Index cutoff of 40 for condition A, condition C, and condition A or C were 56% and 99%, 13% and 99%, and 28% and 99%, respectively.

Conclusions: A model consisting of simultaneous vital sign abnormalities and the pediatric Rothman Index predict condition A or C in the 24-hour period prior to the event. Vital sign only prediction models have higher sensitivity than the pediatric Rothman Index but are associated with a high false-positive rate. The high specificity of the pediatric Rothman Index merits prospective evaluation as an electronic adjunct to human-triggered early warning systems.