Evaluation of the Effectiveness of a Surgical Checklist in Medicare Patients

Abstract

Background: Surgical checklists are increasingly used to improve compliance with evidence-based processes in the perioperative period. Although enthusiasm exists for using checklists to improve outcomes, recent studies have questioned their effectiveness in large populations.

Objective: We sought to examine the association of Keystone Surgery, a statewide implementation of an evidence-based checklist and Comprehensive Unit-based Safety Program, on surgical outcomes and health care costs.

Methods: We performed a study using national Medicare claims data for patients undergoing general and vascular surgery (n=1,002,241) from 2006 to 2011. A difference-in-differences approach was used to evaluate whether implementation was associated with improved surgical outcomes and decreased costs when compared with a national cohort of nonparticipating hospitals. Propensity score matching was used to select 10 control hospitals for each participating hospital. Costs were assessed using price-standardized 30-day Medicare payments for acute hospitalizations, readmissions, and high-cost outliers.

Results: Keystone Surgery implementation in participating centers (N=95 hospitals) was not associated with improved outcomes. Difference-in-differences analysis accounting for trends in nonparticipating hospitals (N=950 hospitals) revealed no differences in adjusted rates of 30-day mortality [relative risk (RR)=1.03; 95% confidence intervals (CI), 0.97-1.10], any complication (RR=1.03; 95% CI, 0.99-1.07), reoperations (RR=0.89; 95% CI, 0.56-1.22), or readmissions (RR=1.01; 95% CI, 0.97-1.05). Medicare payments for the index admission increased following implementation ($516 average increase in payments; 95% CI, $210-$823 increase), as did readmission payments ($564 increase; 95% CI, $89-$1040 increase). High-outlier payments ($965 increase; 95% CI, $974decrease to $2904 increase) did not change.

Conclusions: Implementation of Keystone Surgery in Michigan was not associated with improved outcomes or decreased costs in Medicare patients.