Are population-based diabetes models useful for individual risk estimation?

Abstract

Background: Predictive models are increasingly used in guidelines and informed decision-making interventions. We compared predictions from 2 prominent models for diabetes: the United Kingdom Prospective Diabetes Study (UKPDS) outcomes model and the Archimedes-based Diabetes Personal Health Decisions (PHD) model.

Methods: Ours was a simulation study comparing 10-year and 20-year model predictions for risks of myocardial infarction (MI), stroke, amputation, blindness, and renal failure for representative test cases.

Results: The Diabetes PHD model predicted substantially higher risks of MI and stroke in most cases, particularly for stroke and for 20-year outcomes. In contrast, the UKPDS model predicted risks of amputation and blindness ranging from 2-fold to infinitely higher than the Diabetes PHD model. Predictions for renal failure all differed by more than 2-fold but in a complicated pattern varying by time frame and specific risk factors. Relative to their predictions for white men, the UKPDS model predicted much lower MI and stroke risks for women and Afro-Caribbean men than the Diabetes PHD model did for women and black men. A substantial majority of the Diabetes PHD point estimates fell outside of the UKPDS outcomes model's 95% CIs.

Conclusions: These models produced markedly different predictions. Patients and providers considering risk estimates from such models need to understand their substantial uncertainty and risk of misclassification.