Calibration of clinical prediction rules does not just assess bias

J Clin Epidemiol. 2013 Nov;66(11):1296-301. doi: 10.1016/j.jclinepi.2013.06.003. Epub 2013 Sep 8.


Objectives: Calibration is often thought to assess the bias of a clinical prediction rule. In particular, if the rule is based on a linear logistic model, it is often assumed that an overestimation of all coefficients results in a calibration slope less than 1 and an underestimation in a slope larger than 1.

Study design and setting: We investigate the relation of the bias and the residual variation of clinical prediction rules with the typical behavior of calibration plots and calibration slopes, using some artificial examples.

Results: Calibration is not only sensitive to the bias of the clinical prediction rule but also to the residual variation. In some circumstances, the effects may cancel out, resulting in a misleading perfect calibration.

Conclusion: Poor calibration is a clear indication of limited usefulness of a clinical prediction rule. However, a perfect calibration should be interpreted with care as this may happen even for a biased prediction rule.

Keywords: Bias; Calibration; External validation; Prognosis; Prognostic model; Residual variation.

MeSH terms

  • Bias*
  • Calibration / standards*
  • Decision Support Techniques*
  • Humans
  • Models, Statistical