Misclassification Risk of Tier-Based Physician Quality Performance Systems

Abstract

Objective: There is increasing interest in identifying high-quality physicians, such as whether physicians perform above or below a threshold level. To evaluate whether current methods accurately distinguish above- versus below-threshold physicians, we estimate misclassification rates for two-category identification systems.

Data sources: Claims data for Medicare fee-for-service beneficiaries residing in Florida or New York in 2010.

Study design: Estimate colorectal cancer, glaucoma, and diabetes quality scores for 23,085 physicians. Use a beta-binomial model to estimate physician score reliabilities. Compute the proportion of physicians whose performance tier would be misclassified under three scoring systems.

Principal findings: In the three scoring systems, misclassification ranges were 8.6-25.7 percent, 6.4-22.8 percent, and 4.5-21.7%. True positive rate ranges were 72.9-97.0 percent, 83.4-100.0 percent, and 34.7-88.2 percent. True negative rate ranges were 68.5-91.6 percent, 10.5-92.4 percent, and 81.1-99.9 percent. Positive predictive value ranges were 70.5-91.6 percent, 77.0-97.3 percent, and 55.2-99.1 percent.

Conclusions: Current methods for profiling physicians on quality may produce misleading results, as the number of eligible events is typically small. Misclassification is a policy-relevant measure of the potential impact of tiering on providers, payers, and patients. Quantifying misclassification rates should inform the construction of high-performance networks and quality improvement initiatives.

Keywords: High-performance networks; misclassification; provider profiling; quality of care; reliability.

Figure 1

Examples of Beta Distributions

Figure 2

Distribution of Physician Performance Derived from the Beta‐Binomial Models of Table 2 for Colorectal Cancer (CRC), Glaucoma Screening (GSO), and Diabetes Measures (DIABETES) for Florida and New York Note. Vertical lines represent the mean, 25th, and 75th percentile cut points under the mean‐based high‐performing, broad network with exclusions, or narrow high‐performing network scoring systems, respectively.