Comparative performance of the two pooled cohort equations for predicting atherosclerotic cardiovascular disease

Abstract

Background and aims: Multivariable algorithms have been developed to predict the risk of atherosclerotic cardiovascular disease (ASCVD) to identify high-risk patients. Shortly after the introduction of the AHA/ACC Pooled Cohort Equations (PCE), a systematic overestimation of risk was identified. As such, a revised PCE was proposed to more accurately assess ASCVD risk. This study aims to compare the accuracy of both PCE in predicting ASCVD risk within a large, real-world patient sample in the US.

Methods: This retrospective cohort study identified 20,843 patients aged between 40 and 75 years with no previous ASCVD in an academic healthcare system. Model fit, calibration, and discrimination were compared between PCE using Bayesian Information Criterion (BIC), Hosmer-Lemeshow test, area under the ROC curves (AUC), Brier score, and precision-recall analysis. In addition, we examined race and sex subgroups for effect modification.

Results: Both PCE showed poor calibration (Hosmer-Lemeshow χ² > 20; p < 0.05) and discrimination (AUC<0.7). The lack of improvement in discrimination of the revised PCE (AUC: 0.677 vs 0.679; p = 0.357) was confirmed with the AUC precision-recall curves (AUC_PR: 0.0717 vs 0.0698). In contrast, the AHA/ACC PCE showed a strong positive risk prediction (ΔBIC>10) compared to the revised PCE, although calibration curves had overlapped.

Conclusions: In this single center analysis, both PCE had poor calibration and discrimination of ASCVD risk in a large, real-world patient sample followed up for over 2 years. There was no evidence of improvement in the accuracy of the revised PCE in assessing the risk of ASCVD in relation to the AHA/ACC PCE.

Keywords: Accuracy; Atherosclerotic cardiovascular disease; Risk score assessments.

Figure 1.

Calibration plots with linear prediction of the 10-year ASCVD risk probability across the ASCVD incident events per 1,000 person-years for the AHA/ACC PCE (black squares, dashed line) and the revised PCE (grey circles, solid line) in the overall cohort divided into 25 (A) and 50 quantiles (B) of risk. The reference line (dotted) is when the predicted risk is equal to the observed risk, and has a slope (m) equal to 1 and angle (θ) equal to 45°. ASCVD: atherosclerotic cardiovascular disease; AHA/ACC: American Heart Association/American College of Cardiology; PCE: Pooled Cohort Equations.

Figure 1.

Calibration plots with linear prediction of the 10-year ASCVD risk probability across the ASCVD incident events per 1,000 person-years for the AHA/ACC PCE (black squares, dashed line) and the revised PCE (grey circles, solid line) in the overall cohort divided into 25 (A) and 50 quantiles (B) of risk. The reference line (dotted) is when the predicted risk is equal to the observed risk, and has a slope (m) equal to 1 and angle (θ) equal to 45°. ASCVD: atherosclerotic cardiovascular disease; AHA/ACC: American Heart Association/American College of Cardiology; PCE: Pooled Cohort Equations.

Figure 2.

Comparative precision-recall curves for discrimination power of 10-year ASCVD risk scores assessed using the AHA/ACC PCE (black line) and the revised PCE (grey line).