Automatic classification and prioritisation of actionable BI-RADS categories using natural language processing models

Abstract

Aim: To facilitate the routine tasks performed by radiologists in their evaluation of breast radiology reports, by enhancing the communication of relevant results to referring physicians via a natural language processing (NLP)-based system to classify and prioritise Breast Imaging Reporting Data System (BI-RADS).

Materials and methods: A NLP-based system was developed to classify and prioritise BI-RADS categories from breast ultrasound and mammogram reports, with the potential to streamline and speed up the standard procedures that radiologists must follow to evaluate and categorise breast imaging results. BI-RADS category extraction was divided into two specific tasks: (1) multi-label classification of BI-RADS categories (0-6) and (2) classification of high-priority (BI-RADS 0, 3, 4 and 5) and low priority (BI-RADS 1, 2, and 6) reports according to the previous BI-RADS assessment.

Results: To develop the NLP tool, three different Bidirectional Encoder Representations from Transformers (BERT)-based models (XLM-RoBERTa, BETO, and Bio-BERT-Spanish) were trained and tested on three distinct corpora (containing only breast ultrasound reports, only mammogram reports, or both), and achieved an accuracy of 74.29-77.5% in detecting BI-RADS categories and 88.52-91.02% in prioritising reports.

Conclusion: The system designed can effectively classify all BI-RADS categories present in a single radiology report. In the clinical setting, such an automated tool can assist radiologists in evaluating breast radiology reports and decision-making tasks and enhance the speed of communicating priority BI-RADS reports to referring physicians.