Multicategory reclassification statistics for assessing improvements in diagnostic accuracy

Biostatistics. 2013 Apr;14(2):382-94. doi: 10.1093/biostatistics/kxs047. Epub 2012 Nov 28.


In this paper, we extend the definitions of the net reclassification improvement (NRI) and the integrated discrimination improvement (IDI) in the context of multicategory classification. Both measures were proposed in Pencina and others (2008. Evaluating the added predictive ability of a new marker: from area under the receiver operating characteristic (ROC) curve to reclassification and beyond. Statistics in Medicine 27, 157-172) as numeric characterizations of accuracy improvement for binary diagnostic tests and were shown to have certain advantage over analyses based on ROC curves or other regression approaches. Estimation and inference procedures for the multiclass NRI and IDI are provided in this paper along with necessary asymptotic distributional results. Simulations are conducted to study the finite-sample properties of the proposed estimators. Two medical examples are considered to illustrate our methodology.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Area Under Curve
  • Biomarkers / analysis
  • Biostatistics
  • Computer Simulation
  • Diagnostic Tests, Routine / statistics & numerical data*
  • Gene Expression
  • Humans
  • Leukemia / classification
  • Leukemia / genetics
  • Logistic Models
  • Models, Statistical*
  • ROC Curve
  • Synovitis / diagnosis


  • Biomarkers