Surrounded by quality metrics: what do surgeons think of ACS-NSQIP?

Abstract

Background: In an era of proliferating systems of quality assessment, surgeon confidence in metric tools is essential for successful initiatives in quality improvement. We evaluated surgeons' awareness and attitudes about ACS-NSQIP, which is the only national, surgeon-developed, risk-adjusted, system of surgical outcome assessment.

Methods: A 33-item survey instrument was constructed and content validity established through content expert review; test-retest reliability was assessed (weighted-kappa = 0.72). Survey administration occurred in three institutions with varying ACS-NSQIP experience. Summary statistics were generated and subgroup analyses performed (Fisher's exact test).

Results: One-hundred and eight surgeons participated. Practice experience varied (27% residents, 33% < 10, 12% 10-20, and 28% > 20 years). Seventy-two percent had fellowship training. Surgeons were familiar with ACS-NSQIP structure, including prospective data collection (70%), case-sampling (63%), and reporting as observed/expected ratios (83%). Surgeons knew some collected data-points but misidentified EKG-findings of MI (67%), surgeon case-experience (41%), and anastomotic dehiscence (79%). Most felt ACS-NSQIP would improve quality of care (79%) and identify areas for improvement (92%). Surgeons were less confident regarding utility at an individual level, with only 46% believing surgeon-specific outcomes should be reported. Few thought ACS-NSQIP data should be available publicly (45%), used for marketing (26%), or direct pay-for-performance (24%). Reservations were most pronounced among surgeons with institutional ACS-NSQIP experience.

Conclusion: While surgeons accept ACS-NSQIP at an institutional level, skepticism remains surrounding measurement of individual outcomes and public reporting. Surgeons at institutions with a longer duration of experience with ACS-NSQIP tended to be more cynical about potential data applications. Ongoing education and assessment of surgeons' perceptions of quality improvement initiatives is necessary to ensure surgeons remain engaged actively in determining how quality of care data is measured and utilized.