Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2011 Aug;213(2):231-5.
doi: 10.1016/j.jamcollsurg.2011.04.004. Epub 2011 May 31.

Reliability of superficial surgical site infections as a hospital quality measure

Lillian S Kao  1 , Amir A Ghaferi, Clifford Y Ko, Justin B Dimick
Affiliations

Reliability of superficial surgical site infections as a hospital quality measure

Lillian S Kao et al. J Am Coll Surg. 2011 Aug.

Abstract

Background: Although rates of superficial surgical site infection (SSI) are increasingly used as measures of hospital quality, the statistical reliability of using SSI rates in this context is uncertain. We used the American College of Surgeons National Surgical Quality Improvement Program data to determine the reliability of SSI rates as a measure of hospital performance and to evaluate the effect of hospital caseload on reliability.

Study design: We examined all patients who underwent colon resection in hospitals participating in the American College of Surgeons National Surgical Quality Improvement Program in 2007 (n = 18,455 patients, n = 181 hospitals). We first calculated the number of cases and the risk-adjusted rate of SSI at each hospital. We then used hierarchical modeling to estimate the reliability of this quality measure for each hospital. Finally, we quantified the proportion of hospital-level variation in SSI rates due to patient characteristics and measurement noise.

Results: The average number of colon resections per hospital was 102 (SD 65). The risk-adjusted rate of superficial SSI was 10.5%, but varied from 0 to 30% across hospitals. Approximately 35% of the variation in SSI rates was explained by noise, 7% could be attributed to patient characteristics, and the remaining 58% represented true differences in SSI rates. Just more than half of the hospitals (54%) had a reliability >0.70, which is considered a minimum acceptable level. To achieve this level of reliability, 94 cases were required.

Conclusions: SSI rates are a reliable measure of hospital quality when an adequate number of cases have been reported. For hospitals with inadequate caseloads, the National Surgical Quality Improvement Program sampling strategy could be altered to provide enough cases to ensure reliability.

PubMed Disclaimer

Figures

Figure 1
Figure 1
The proportion of superficial SSIs after colon resections that are attributable to patient factors, “noise” or measurement error, and hospital performance.
Figure 2
Figure 2
Relationship between reliability and hospital caseload of colon resections based on the ACS NSQIP 2007 database.
See this image and copyright information in PMC

Comment in

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Anderson DJ, Chen LF, Sexton DJ, Kaye KS. Complex surgical site infections and the devilish details of risk adjustment: important implications for public reporting. Infect Control Hosp Epidemiol. 2008;29:941–6. - PubMed
    1. Brandt C, Hansen S, Sohr D, Daschner F, Ruden H, Gastmeier P. Finding a method for optimizing risk adjustment when comparing surgical-site infection rates. Infect Control Hosp Epidemiol. 2004;25:313–8. - PubMed
    1. Nosocomial infection rates for interhospital comparison: limitations and possible solutions. A Report from the National Nosocomial Infections Surveillance (NNIS) System. Infect Control Hosp Epidemiol. 1991;12:609–21. - PubMed
    1. Dimick JB, Welch HG, Birkmeyer JD. Surgical mortality as an indicator of hospital quality: the problem with small sample size. Jama. 2004;292:847–51. - PubMed
    1. Shiloach M, Frencher SK, Jr., Steeger JE, et al. Toward robust information: data quality and inter-rater reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg. 2010;210:6–16. - PubMed

Publication types