Accuracy of a computerized clinical decision-support system for asthma assessment and management

Abstract

Objective: To evaluate the accuracy of a computerized clinical decision-support system (CDSS) designed to support assessment and management of pediatric asthma in a subspecialty clinic.

Design: Cohort study of all asthma visits to pediatric pulmonology from January to December, 2009.

Measurements: CDSS and physician assessments of asthma severity, control, and treatment step.

Results: Both the clinician and the computerized CDSS generated assessments of asthma control in 767/1032 (74.3%) return patients, assessments of asthma severity in 100/167 (59.9%) new patients, and recommendations for treatment step in 66/167 (39.5%) new patients. Clinicians agreed with the CDSS in 543/767 (70.8%) of control assessments, 37/100 (37%) of severity assessments, and 19/66 (29%) of step recommendations. External review classified 72% of control disagreements (21% of all control assessments), 56% of severity disagreements (37% of all severity assessments), and 76% of step disagreements (54% of all step recommendations) as CDSS errors. The remaining disagreements resulted from pulmonologist error or ambiguous guidelines. Many CDSS flaws, such as attributing all 'cough' to asthma, were easily remediable. Pediatric pulmonologists failed to follow guidelines in 8% of return visits and 18% of new visits.

Limitations: The authors relied on chart notes to determine clinical reasoning. Physicians may have changed their assessments after seeing CDSS recommendations.

Conclusions: A computerized CDSS performed relatively accurately compared to clinicians for assessment of asthma control but was inaccurate for treatment. Pediatric pulmonologists failed to follow guideline-based care in a small proportion of patients.

Figure 1

Data-entry form to capture impairment and risk, used to calculate asthma severity during new patient visits. CC, chief complaint; EIB, exercise-induced bronchoconstriction; ER, emergency room; FEV, forced expiratory volume; FEV1, forced expiratory volume in 1 s; FVC, forced vital capacity; NHLBI, National Heart Lung and Blood Institute; ROS, review of systems; SABA, short-acting β2-agonist; wk, week; YNHH, Yale-New Haven Hospital; yr, years.

Figure 2

Data-entry form to capture impairment and risk, used to calculate asthma control during return patient visits. ACT, asthma control test; EIB, exercise-induced bronchoconstriction; ER, emergency room; FEV1, forced expiratory volume in 1 s; FVC, forced vital capacity; ROS, review of systems; SABA, short-acting β2-agonist; wk, week.

Figure 3

Asthma assessment form used for new patients; return patient form is similar but replaces decision support severity assessment with control assessment. CC, chief complaint; ROS, review of systems.

Figure 4

Treatment step recommendation screen. Note that the provider has selected step 6, which prompted the alert to appear at the top suggesting step 3 instead. COMBO, combination inhaler containing both ICS and LABA; ICS, inhaled corticosteroid; LABA, long-acting beta agonist; LTRA, leukotriene receptor antagonist; NHLBI, National Heart Lung and Blood Institute; PRN, as needed.