Comparing diagnostic tests on benefit-risk

J Biopharm Stat. 2016;26(6):1083-1097. doi: 10.1080/10543406.2016.1226335. Epub 2016 Aug 22.


Comparing diagnostic tests on accuracy alone can be inconclusive. For example, a test may have better sensitivity than another test yet worse specificity. Comparing tests on benefit risk may be more conclusive because clinical consequences of diagnostic error are considered. For benefit-risk evaluation, we propose diagnostic yield, the expected distribution of subjects with true positive, false positive, true negative, and false negative test results in a hypothetical population. We construct a table of diagnostic yield that includes the number of false positive subjects experiencing adverse consequences from unnecessary work-up. We then develop a decision theory for evaluating tests. The theory provides additional interpretation to quantities in the diagnostic yield table. It also indicates that the expected utility of a test relative to a perfect test is a weighted accuracy measure, the average of sensitivity and specificity weighted for prevalence and relative importance of false positive and false negative testing errors, also interpretable as the cost-benefit ratio of treating non-diseased and diseased subjects. We propose plots of diagnostic yield, weighted accuracy, and relative net benefit of tests as functions of prevalence or cost-benefit ratio. Concepts are illustrated with hypothetical screening tests for colorectal cancer with test positive subjects being referred to colonoscopy.

Keywords: Clinical utility; cost/benefit ratio; decision theory; diagnostic yield; relative net benefit; risk threshold; weighted accuracy.

Publication types

  • Comparative Study

MeSH terms

  • Colonoscopy
  • Colorectal Neoplasms / diagnosis
  • Diagnostic Tests, Routine*
  • False Negative Reactions
  • False Positive Reactions
  • Humans
  • Prevalence
  • Risk Assessment*
  • Sensitivity and Specificity