Lessons learnt when accounting for competing events in the external validation of time-to-event prognostic models

Int J Epidemiol. 2022 May 9;51(2):615-625. doi: 10.1093/ije/dyab256.

Abstract

Background: External validation of prognostic models is necessary to assess the accuracy and generalizability of the model to new patients. If models are validated in a setting in which competing events occur, these competing risks should be accounted for when comparing predicted risks to observed outcomes.

Methods: We discuss existing measures of calibration and discrimination that incorporate competing events for time-to-event models. These methods are illustrated using a clinical-data example concerning the prediction of kidney failure in a population with advanced chronic kidney disease (CKD), using the guideline-recommended Kidney Failure Risk Equation (KFRE). The KFRE was developed using Cox regression in a diverse population of CKD patients and has been proposed for use in patients with advanced CKD in whom death is a frequent competing event.

Results: When validating the 5-year KFRE with methods that account for competing events, it becomes apparent that the 5-year KFRE considerably overestimates the real-world risk of kidney failure. The absolute overestimation was 10%age points on average and 29%age points in older high-risk patients.

Conclusions: It is crucial that competing events are accounted for during external validation to provide a more reliable assessment the performance of a model in clinical settings in which competing risks occur.

Keywords: Prediction; calibration; competing risks; discrimination; external validation; prognostic model.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Aged
  • Female
  • Humans
  • Male
  • Prognosis
  • Renal Insufficiency*
  • Renal Insufficiency, Chronic* / epidemiology
  • Risk Assessment / methods