Effect of cancer prevalence on the use of risk-assessment cut-off levels and the performance of mathematical models to distinguish malignant from benign adnexal masses

Abstract

Objective: Two logistic regression models have been developed for the characterization of adnexal masses. The goal of this prospective analysis was to see whether these models perform differently according to the prevalence of malignancy and whether the cut-off levels of risk assessment for malignancy by the models require modification in different centers.

Methods: Centers were categorized into those with a prevalence of malignancy below 15%, between 15 and 30% and above 30%. The areas under the receiver-operating characteristics curves (AUC) were compared using bootstrapping. The optimal cut-off level of risk assessment for malignancy was chosen per center, corresponding to the highest sensitivity level possible while still keeping a good specificity.

Results: Both models performed better in centers with a lower prevalence of malignant cases. The AUCs of the two models for centers with fewer than 15% malignant cases were 0.97 and 0.95, those of centers with 15-30% malignancy were 0.95 and 0.93 and those of centers with more than 30% malignant cases were 0.94 and 0.92. This decrease in performance was due mainly to the decrease in specificity from over 90 to around 76%. In the centers with a higher percentage of malignant cases, a sensitivity of at least 90% with a good specificity could not be obtained by choosing a different cut-off level.

Conclusions: Overall the models performed well in all centers. The performance of the logistic regression models worsened with increasing prevalence of malignancy, due to a case mix with more borderline and complex benign masses seen in those centers. Because the cut-off of 0.10 is optimal for all three types of center, it seems reasonable to use this cut-off for both models in all centers.