Retrospective evaluation of a CE-marked AI system, including 1,017,208 mammography screening examinations

Tone Hovda; Marthe Larsen; Marie Burns Bergan; Jonas Gjesvik; Lars A Akslen; Solveig Hofvind

doi:10.1007/s00330-025-11521-4

Retrospective evaluation of a CE-marked AI system, including 1,017,208 mammography screening examinations

Eur Radiol. 2025 Sep;35(9):5685-5694. doi: 10.1007/s00330-025-11521-4. Epub 2025 Mar 26.

Authors

Tone Hovda¹, Marthe Larsen², Marie Burns Bergan², Jonas Gjesvik², Lars A Akslen^{3

4}, Solveig Hofvind^{5

6}

Affiliations

¹ Department of Radiology, Vestre Viken Hospital Trust, Drammen, Norway.
² Section for Breast Cancer Screening, Cancer Registry of Norway, Norwegian Institute of Public Health, Oslo, Norway.
³ Centre for Cancer Biomarkers CCBIO, Department of Clinical Medicine, Section for Pathology, University of Bergen, Bergen, Norway.
⁴ Department of Pathology, Haukeland University Hospital, Bergen, Norway.
⁵ Section for Breast Cancer Screening, Cancer Registry of Norway, Norwegian Institute of Public Health, Oslo, Norway. sshh@kreftregisteret.no.
⁶ Department of Health and Care Sciences, Faculty of Health Sciences, UiT The Arctic University of Norway, Tromsø, Norway. sshh@kreftregisteret.no.

Abstract

Objectives: To retrospectively evaluate the performance of a CE-marked AI system for identifying breast cancer on screening mammograms. Evidence from large retrospective studies is crucial for planning prospective studies and to further ensure safe implementation.

Materials and methods: We used data from screening examinations performed from 2004 to 2021 at ten breast centers in BreastScreen Norway. In the standard independent double reading setting, each radiologist scored each breast from 1 (negative) to 5 (high probability of cancer). The AI system assigned each examination an NT and an SN score; the NT score aimed to classify examinations as negative with minimal misclassification while the SN score aimed to classify examinations as positive with high confidence. N70 was defined as being among the 70% with the lowest NT score and P3 was defined as being among the 3% with the highest SN score.

Results: A total of 1,017,208 screening examinations were included in the study sample. At N70, 1.8% (107/5977) of the screen-detected and 34.5% (625/1812) of the interval cancers were defined as negative. Using P3 to define cases as positive, 81.5% (4871/5977) of the screen-detected and 19.0% (344/1812) of the interval cancers were defined as positive. Among the screen-detected cancers in N70, 11.2% (12/107) had an interpretation score > 2 by both radiologists.

Conclusion: The AI system performed well according to identifying negative cases and cancer cases. Thus, the AI system can be used to reduce workload for the radiologists and potentially increase the sensitivity of mammography.

Key points: Question Results from large mammography screening samples not used in training AI algorithms are important to consider when planning prospective studies and implementation. Findings More than 80% of the screening-detected cancers were classified as positive by AI when considering 3% of the examinations with the highest AI risk score as positive. Clinical relevance A lack of radiologists is a challenge in mammographic screening. Our findings support other studies that suggest the use of AI to reduce screen-reading workload.

Keywords: Artificial intelligence; Breast cancer; Mammography; Screening.

Publication types

Multicenter Study

MeSH terms

Adult
Aged
Artificial Intelligence*
Breast Neoplasms* / diagnostic imaging
Early Detection of Cancer* / methods
Female
Humans
Mammography* / methods
Middle Aged
Norway
Radiographic Image Interpretation, Computer-Assisted* / methods
Retrospective Studies
Sensitivity and Specificity

Grants and funding

214931/Kreftforeningen