Diagnostic efficiency of existing guidelines and the AI-SONIC™ artificial intelligence for ultrasound-based risk assessment of thyroid nodules

Abstract

Introduction: The thyroid ultrasound guidelines include the American College of Radiology Thyroid Imaging Reporting and Data System, Chinese-Thyroid Imaging Reporting and Data System, Korean Society of Thyroid Radiology, European-Thyroid Imaging Reporting and Data System, American Thyroid Association, and American Association of Clinical Endocrinologists/American College of Endocrinology/Associazione Medici Endocrinologi guidelines. This study aimed to compare the efficiency of the six ultrasound guidelines vs. an artificial intelligence system (AI-SONICTM) in differentiating thyroid nodules, especially medullary thyroid carcinoma.

Methods: This retrospective study included patients with medullary thyroid carcinoma, papillary thyroid carcinoma, or benign nodules who underwent nodule resection between May 2010 and April 2020 at one hospital. The diagnostic efficacy of the seven diagnostic tools was evaluated using the receiver operator characteristic curves.

Results: Finally, 432 patients with 450 nodules were included for analysis. The American Association of Clinical Endocrinologists/American College of Endocrinology/Associazione Medici Endocrinologi guidelines had the best sensitivity (88.1%) and negative predictive value (78.6%) for differentiating papillary thyroid carcinoma or medullary thyroid carcinoma vs. benign nodules, while the Korean Society of Thyroid Radiology guidelines had the best specificity (85.6%) and positive predictive value (89.6%), and the American Thyroid Association guidelines had the best accuracy (83.7%). When assessing medullary thyroid carcinoma, the American Thyroid Association guidelines had the highest area under the curve (0.78), the American College of Radiology Thyroid Imaging Reporting and Data System guidelines had the best sensitivity (90.2%), and negative predictive value (91.8%), and AI-SONICTM had the best specificity (85.6%) and positive predictive value (67.5%). The Chinese-Thyroid Imaging Reporting and Data System guidelines had the best under the curve (0.86) in diagnosing malignant tumors vs. benign tumors, followed by the American Thyroid Association and Korean Society of Thyroid Radiology guidelines. The best positive likelihood ratios were achieved by the Korean Society of Thyroid Radiology guidelines and AI-SONICTM (both 5.37). The best negative likelihood ratio was achieved by the American Association of Clinical Endocrinologists/American College of Endocrinology/Associazione Medici Endocrinologi guidelines (0.17). The highest diagnostic odds ratio was achieved by the American Thyroid Association guidelines (24.78).

Discussion: All six guidelines and the AI-SONICTM system had satisfactory value in differentiating benign vs. malignant thyroid nodules.

Keywords: artificial intelligence; medullary thyroid carcinoma; papillary thyroid carcinoma; thyroid nodule; ultrasonography.