Variability of interpretive accuracy among diagnostic mammography facilities

Abstract

Background: Interpretive performance of screening mammography varies substantially by facility, but performance of diagnostic interpretation has not been studied.

Methods: Facilities performing diagnostic mammography within three registries of the Breast Cancer Surveillance Consortium were surveyed about their structure, organization, and interpretive processes. Performance measurements (false-positive rate, sensitivity, and likelihood of cancer among women referred for biopsy [positive predictive value of biopsy recommendation {PPV2}]) from January 1, 1998, through December 31, 2005, were prospectively measured. Logistic regression and receiver operating characteristic (ROC) curve analyses, adjusted for patient and radiologist characteristics, were used to assess the association between facility characteristics and interpretive performance. All statistical tests were two-sided.

Results: Forty-five of the 53 facilities completed a facility survey (85% response rate), and 32 of the 45 facilities performed diagnostic mammography. The analyses included 28 100 diagnostic mammograms performed as an evaluation of a breast problem, and data were available for 118 radiologists who interpreted diagnostic mammograms at the facilities. Performance measurements demonstrated statistically significant interpretive variability among facilities (sensitivity, P = .006; false-positive rate, P < .001; and PPV2, P < .001) in unadjusted analyses. However, after adjustment for patient and radiologist characteristics, only false-positive rate variation remained statistically significant and facility traits associated with performance measures changed (false-positive rate = 6.5%, 95% confidence interval [CI] = 5.5% to 7.4%; sensitivity = 73.5%, 95% CI = 67.1% to 79.9%; and PPV2 = 33.8%, 95% CI = 29.1% to 38.5%). Facilities reporting that concern about malpractice had moderately or greatly increased diagnostic examination recommendations at the facility had a higher false-positive rate (odds ratio [OR] = 1.48, 95% CI = 1.09 to 2.01) and a non-statistically significantly higher sensitivity (OR = 1.74, 95% CI = 0.94 to 3.23). Facilities offering specialized interventional services had a non-statistically significantly higher false-positive rate (OR = 1.97, 95% CI = 0.94 to 4.1). No characteristics were associated with overall accuracy by ROC curve analyses.

Conclusions: Variation in diagnostic mammography interpretation exists across facilities. Failure to adjust for patient characteristics when comparing facility performance could lead to erroneous conclusions. Malpractice concerns are associated with interpretive performance.

Figure 1

Study populations included in the analyses, including number of facilities, radiologists, patients, mammograms, and cancers. *CO = Colorado; NH = New Hampshire; WA = Washington.

Figure 2

Unadjusted diagnostic mammography performance measures for the 32 facilities. Overall mean at the facility level indicated by a line. A) False-positive rate. The unadjusted facility-level mean was 6.5% (95% CI = 5.5 to 7.4). B) Sensitivity. The unadjusted facility-level mean was 73.5% (95% CI = 67.1 to 79.9); facility no. 32 was without any cancers during follow-up. C) PPV2. The unadjusted facility-level mean was 33.8% (95% CI = 29.1 to 38.5). Overall mean at the facility level was computed by calculating the performance measures for each facility and taking the average across facilities. Each facility was given the same weight regardless of the number of screens interpreted at the facility. Diamonds indicate mean values; error bars correspond to 95% CIs. Facilities are ordered by number of diagnostic evaluations (highest to lowest) included in the analysis. Statistically significant variability was found across the facilities in unadjusted analyses for all performance measures (P < .01, calculated using two-sided F tests). CI = confidence intervals; PPV2 = positive predictive value of biopsy.