Automated coronary artery calcification scoring in non-gated chest CT: agreement and reliability

PLoS One. 2014 Mar 13;9(3):e91239. doi: 10.1371/journal.pone.0091239. eCollection 2014.


Objective: To determine the agreement and reliability of fully automated coronary artery calcium (CAC) scoring in a lung cancer screening population.

Materials and methods: 1793 low-dose chest CT scans were analyzed (non-contrast-enhanced, non-gated). To establish the reference standard for CAC, first automated calcium scoring was performed using a preliminary version of a method employing coronary calcium atlas and machine learning approach. Thereafter, each scan was inspected by one of four trained raters. When needed, the raters corrected initially automaticity-identified results. In addition, an independent observer subsequently inspected manually corrected results and discarded scans with gross segmentation errors. Subsequently, fully automatic coronary calcium scoring was performed. Agatston score, CAC volume and number of calcifications were computed. Agreement was determined by calculating proportion of agreement and examining Bland-Altman plots. Reliability was determined by calculating linearly weighted kappa (κ) for Agatston strata and intraclass correlation coefficient (ICC) for continuous values.

Results: 44 (2.5%) scans were excluded due to metal artifacts or gross segmentation errors. In the remaining 1749 scans, median Agatston score was 39.6 (P25-P75∶0-345.9), median volume score was 60.4 mm3 (P25-P75∶0-361.4) and median number of calcifications was 2 (P25-P75∶0-4) for the automated scores. The κ demonstrated very good reliability (0.85) for Agatston risk categories between the automated and reference scores. The Bland-Altman plots showed underestimation of calcium score values by automated quantification. Median difference was 2.5 (p25-p75∶0.0-53.2) for Agatston score, 7.6 (p25-p75∶0.0-94.4) for CAC volume and 1 (p25-p75∶0-5) for number of calcifications. The ICC was very good for Agatston score (0.90), very good for calcium volume (0.88) and good for number of calcifications (0.64).

Discussion: Fully automated coronary calcium scoring in a lung cancer screening setting is feasible with acceptable reliability and agreement despite an underestimation of the amount of calcium when compared to reference scores.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Aged
  • Coronary Artery Disease / diagnostic imaging
  • Coronary Vessels / diagnostic imaging
  • Coronary Vessels / physiopathology*
  • Diagnosis, Computer-Assisted
  • False Negative Reactions
  • Female
  • Humans
  • Lung Neoplasms / complications*
  • Lung Neoplasms / diagnosis*
  • Male
  • Middle Aged
  • Observer Variation
  • Pattern Recognition, Automated
  • Reproducibility of Results
  • Risk
  • Severity of Illness Index
  • Smoking
  • Tomography, X-Ray Computed
  • Vascular Calcification / diagnostic imaging*

Grant support

The NELSON-trial was supported by the Netherlands Organisation for Health Research and Development (ZonMw, grant number 22000130,; Dutch Cancer Society Koningin Wilhelmina Fonds; Stichting Centraal Fonds Reserves van Voormalig Vrijwillige Ziekenfondsverzekeringen (RVVZ); Siemens Germany provided 4 digital workstations and LungCARE for the performance of 3D measurements; Rotterdam Oncologic Thoracic Steering Committee; and the G. Ph. Verhagen Trust, Flemish League Against Cancer, Foundation Against Cancer, and Erasmus Trust Fund. The contribution of I. Isgum was financially partly supported by the project Care4Me (Cooperative Advanced Research for Medical Efficiency, grant number ITEA2 Call3-08004, in the framework of the EU research programme ITEA (Information Technology for European Advancement). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.