Evaluation of surveillance bias and the validity of the venous thromboembolism quality measure

Abstract

Importance: Postoperative venous thromboembolism (VTE) rates are widely reported quality metrics soon to be used in pay-for-performance programs. Surveillance bias occurs when some clinicians use imaging studies to detect VTE more frequently than other clinicians. Because they look more, they find more VTE events, paradoxically worsening their hospital's VTE quality measure performance. A surveillance bias may influence VTE measurement if (1) greater hospital VTE prophylaxis adherence fails to result in lower measured VTE rates, (2) hospitals with characteristics suggestive of higher quality (eg, more accreditations) have greater VTE prophylaxis adherence rates but worse VTE event rates, and (3) higher hospital VTE imaging utilization use rates are associated with higher measured VTE event rates.

Objective: To examine whether a surveillance bias influences the validity of reported VTE rates.

Design, setting, and participants: 2010 Hospital Compare and American Hospital Association data from 2838 hospitals were merged. Next, 2009-2010 Medicare claims data for 954,926 surgical patient discharges from 2786 hospitals who were undergoing 1 of 11 major operations were used to calculate VTE imaging (duplex ultrasonography, chest computed tomography/magnetic resonance imaging, and ventilation-perfusion scans) and VTE event rates.

Main outcomes and measures: The association between hospital VTE prophylaxis adherence and risk-adjusted VTE event rates was examined. The relationship between a summary score of hospital structural characteristics reflecting quality (hospital size, numbers of accreditations/quality initiatives) and performance on VTE prophylaxis and risk-adjusted VTE measures was examined. Hospital-level VTE event rates were compared across VTE diagnostic imaging rate quartiles and with a quantile regression.

Results: Greater hospital VTE prophylaxis adherence rates were weakly associated with worse risk-adjusted VTE event rates (r2 = 4.2%; P = .03). Hospitals with increasing structural quality scores had higher VTE prophylaxis adherence rates (93.3% vs 95.5%, lowest vs highest quality quartile; P < .001) but worse risk-adjusted VTE rates (4.8 vs 6.4 per 1000, lowest vs highest quality quartile; P < .001). Mean VTE diagnostic imaging rates ranged from 32 studies per 1000 in the lowest imaging use quartile to 167 per 1000 in the highest quartile (P < .001). Risk-adjusted VTE rates increased significantly with VTE imaging use rates in a stepwise fashion, from 5.0 per 1000 in the lowest quartile to 13.5 per 1000 in the highest quartile (P < .001).

Conclusions and relevance: Hospitals with higher quality scores had higher VTE prophylaxis rates but worse risk-adjusted VTE rates. Increased hospital VTE event rates were associated with increasing hospital VTE imaging use rates. Surveillance bias limits the usefulness of the VTE quality measure for hospitals working to improve quality and patients seeking to identify a high-quality hospital.