Evaluation of the Effectiveness of a Surgical Checklist in Medicare Patients

Med Care. 2015 Jan;53(1):87-94. doi: 10.1097/MLR.0000000000000277.


Background: Surgical checklists are increasingly used to improve compliance with evidence-based processes in the perioperative period. Although enthusiasm exists for using checklists to improve outcomes, recent studies have questioned their effectiveness in large populations.

Objective: We sought to examine the association of Keystone Surgery, a statewide implementation of an evidence-based checklist and Comprehensive Unit-based Safety Program, on surgical outcomes and health care costs.

Methods: We performed a study using national Medicare claims data for patients undergoing general and vascular surgery (n=1,002,241) from 2006 to 2011. A difference-in-differences approach was used to evaluate whether implementation was associated with improved surgical outcomes and decreased costs when compared with a national cohort of nonparticipating hospitals. Propensity score matching was used to select 10 control hospitals for each participating hospital. Costs were assessed using price-standardized 30-day Medicare payments for acute hospitalizations, readmissions, and high-cost outliers.

Results: Keystone Surgery implementation in participating centers (N=95 hospitals) was not associated with improved outcomes. Difference-in-differences analysis accounting for trends in nonparticipating hospitals (N=950 hospitals) revealed no differences in adjusted rates of 30-day mortality [relative risk (RR)=1.03; 95% confidence intervals (CI), 0.97-1.10], any complication (RR=1.03; 95% CI, 0.99-1.07), reoperations (RR=0.89; 95% CI, 0.56-1.22), or readmissions (RR=1.01; 95% CI, 0.97-1.05). Medicare payments for the index admission increased following implementation ($516 average increase in payments; 95% CI, $210-$823 increase), as did readmission payments ($564 increase; 95% CI, $89-$1040 increase). High-outlier payments ($965 increase; 95% CI, $974decrease to $2904 increase) did not change.

Conclusions: Implementation of Keystone Surgery in Michigan was not associated with improved outcomes or decreased costs in Medicare patients.

Publication types

  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Aged
  • Animals
  • Checklist / statistics & numerical data*
  • Dogs
  • Female
  • Hospital Charges / statistics & numerical data
  • Humans
  • Intraoperative Complications / epidemiology
  • Male
  • Medicare
  • Middle Aged
  • Outcome and Process Assessment, Health Care
  • Postoperative Complications / epidemiology
  • Quality Improvement / organization & administration*
  • Surgical Procedures, Operative / economics
  • Surgical Procedures, Operative / methods*
  • Surgical Procedures, Operative / mortality
  • Surgical Procedures, Operative / statistics & numerical data*
  • United States