A Comparison of the Performances of Artificial Intelligence System and Radiologists in the Ultrasound Diagnosis of Thyroid Nodules

Abstract

Aims: The purpose of this paper is to prospectively evaluate the performance of an artificial intelligence (AI) system in diagnosing thyroid nodules and to assess its potential value in comparison with the performance of radiologists with different levels of experience, as well as the factors affecting its diagnostic accuracy.

Background: In recent years, medical imaging diagnosis using AI has become a popular topic in clinical application research.

Objective: This study aimed to evaluate the performance of an AI system in diagnosing thyroid nodules and compare it with the performance levels of different radiologists.

Methods: This study involved 426 patients screened for thyroid nodules at the First Affiliated Hospital of Guangzhou Medical University between July 2017 and March 2019. All of the nodules were evaluated by radiologists with various levels of experience and an AI system. The diagnostic performances of two junior and two senior radiologists, an AI system, and an AI-assisted junior radiologist were compared, as were their diagnostic results with respect to nodules of different sizes.

Results: The senior radiologists, the AI system, and the AI-assisted junior radiologist performed better than the junior radiologist (p < 0.05). The area under the curves of the AI system and the AI-assisted junior radiologist were similar to the curve of the senior radiologists (p > 0.05). The diagnostic results concerning the two nodule sizes showed that the diagnostic error rates of the AI system, junior radiologists, and senior radiologists for nodules with a maximum diameter of ≤1 cm (Dmax ≤ 1 cm) were higher than those for nodules with a maximum diameter of 1 cm (D_max > 1 cm) (23.4% vs. 12.1%, p = 0.002; 26.6% vs. 7.3%, p < 0.001; and 38.3% vs. 14.6%, p < 0.001).

Conclusion: The AI system is a decision-making tool that could potentially improve the diagnostic efficiency of junior radiologists. Micronodules with Dmax ≤ 1cm were significantly correlated with diagnostic accuracy; accordingly, more micronodules of this size, in particular, should be added to the AI system as training samples. Other: The system could be a potential decision-making tool for effectively improving the diagnostic efficiency of junior radiologists in the community.

Keywords: Artificial intelligence; decision-making; deep learning; diagnosis; thyroid nodule; ultrasound.