An analysis of calibration and discrimination among multiple cardiovascular risk scores in a modern multiethnic cohort

Abstract

Background: Accurate risk assessment of atherosclerotic cardiovascular disease (ASCVD) is essential to effectively balance the risks and benefits of therapy for primary prevention.

Objective: To compare the calibration and discrimination of the new American Heart Association (AHA) and American College of Cardiology (ACC) ASCVD risk score with alternative risk scores and to explore preventive therapy as a cause of the reported risk overestimation using the AHA-ACC-ASCVD score.

Design: Prospective epidemiologic study of ASCVD.

Setting: MESA (Multi-Ethnic Study of Atherosclerosis), a community-based, sex-balanced, multiethnic cohort.

Patients: 4227 MESA participants aged 50 to 74 years and without diabetes at baseline.

Measurements: Observed and expected events for the AHA-ACC-ASCVD score were compared with 4 commonly used risk scores-and their respective end points-in MESA after a 10.2-year follow-up.

Results: The new AHA-ACC-ASCVD and 3 older Framingham-based risk scores overestimated cardiovascular events by 37% to 154% in men and 8% to 67% in women. Overestimation was noted throughout the continuum of risk. In contrast, the Reynolds Risk Score overestimated risk by 9% in men but underestimated risk by 21% in women. Aspirin, lipid-lowering or antihypertensive therapy, and interim revascularization did not explain the overestimation.

Limitation: Comparability of MESA with target populations for primary prevention and possibility of missed events in MESA.

Conclusion: Of the 5 risk scores, 4, including the new AHA-ACC-ASCVD score, showed overestimation of risk (25% to 115%) in a modern, multiethnic cohort without baseline clinical ASCVD. If validated, overestimation of ASCVD risk may have substantial implications for individual patients and the health care system.

Primary funding source: National Heart, Lung, and Blood Institute.

Figure 1

Risk score–specific predicted and observed events in men, by decile of calculated risk. Hosmer–Lemeshow calibration plots for men (n = 1961). ACC = American College of Cardiology; AHA = American Heart Association; ASCVD = atherosclerotic cardiovascular disease; ATPIII = Adult Treatment Panel III; CHD = coronary heart disease; CVD = cardiovascular disease; FRS = Framingham risk score; RRS = Reynolds Risk Score.

Figure 2

Risk score–specific predicted and observed events in women, by decile of calculated risk. Hosmer–Lemeshow calibration plots for women (n = 2266). ACC = American College of Cardiology; AHA = American Heart Association; ASCVD = atherosclerotic cardiovascular disease; ATPIII = Adult Treatment Panel III; CHD = coronary heart disease; CVD = cardiovascular disease; FRS = Framingham risk score; RRS = Reynolds Risk Score.

Appendix Figure

Area under the curve for all 5 risk prediction models. ACC = American College of Cardiology; AHA = American Heart Association; ASCVD = atherosclerotic cardiovascular disease; ATPIII = Adult Treatment Panel III; CHD = coronary heart disease; CVD = cardiovascular disease; FRS = Framingham risk score; RRS = Reynolds Risk Score. Left. Men (n = 1961). Right. Women (n = 2266).