Constructing a composite quality score for the care of acute myocardial infarction patients at discharge: impact on hospital ranking

Abstract

Objective: To determine the impact on hospital ranking of different aggregation methods when creating a composite score from a set of quality indicators relating to a single clinical condition.

Design: The analysis was based on 14966 medical records taken from all French hospitals that treated over 30 patients with acute myocardial infarction in 2008 (n=275). Five quality indicators measuring the quality of care delivered to patients with acute myocardial infarction at hospital discharge were aggregated by 5 methods issued from a variety of activity sectors (indicator average, all-or-none, budget allocation process, benefit of the doubt, and unobserved component model).

Main outcome measures: Each aggregation method was used to rank hospitals into 3 categories depending on the position of the 95% confidence interval of the composite score relative to the overall mean. Variations in rank according to method were estimated using weighted κ coefficients.

Results: Agreement between methods ranged from poor (κ=0.20) to almost perfect (κ=0.84). A change of method led to a change in rank for 71% (196 of 275) of hospitals. Only 14 of 121 hospitals which were ranked top and 20 of 118 which were ranked bottom, by at least 1 of the 5 methods, held their rank on a switch to the 4 other methods.

Conclusion: Hospital ranking varied widely according to 5 aggregation methods. If one method has to be chosen, for instance for reporting to governments, regulatory agencies, payers, health care professionals, and the public, it is necessary to provide its rationale and characteristics, and information on score uncertainty.