Toward automated severe pharyngitis detection with smartphone camera using deep learning networks

Comput Biol Med. 2020 Oct;125:103980. doi: 10.1016/j.compbiomed.2020.103980. Epub 2020 Aug 20.


Purpose: Severe pharyngitis is frequently associated with inflammations caused by streptococcal pharyngitis, which can cause immune-mediated and post-infectious complications. The recent global pandemic of coronavirus disease (COVID-19) encourages the use of telemedicine for patients with respiratory symptoms. This study, therefore, purposes automated detection of severe pharyngitis using a deep learning framework with self-taken throat images.

Methods: A dataset composed of two classes of 131 throat images with pharyngitis and 208 normal throat images was collected. Before the training classifier, we constructed a cycle consistency generative adversarial network (CycleGAN) to augment the training dataset. The ResNet50, Inception-v3, and MobileNet-v2 architectures were trained with transfer learning and validated using a randomly selected test dataset. The performance of the models was evaluated based on the accuracy and area under the receiver operating characteristic curve (ROC-AUC).

Results: The CycleGAN-based synthetic images reflected the pragmatic characteristic features of pharyngitis. Using the synthetic throat images, the deep learning model demonstrated a significant improvement in the accuracy of the pharyngitis diagnosis. ResNet50 with GAN-based augmentation showed the best ROC-AUC of 0.988 for pharyngitis detection in the test dataset. In the 4-fold cross-validation using the ResNet50, the highest detection accuracy and ROC-AUC achieved were 95.3% and 0.992, respectively.

Conclusion: The deep learning model for smartphone-based pharyngitis screening allows fast identification of severe pharyngitis with a potential of the timely diagnosis of pharyngitis. In the recent pandemic of COVID-19, this framework will help patients with upper respiratory symptoms to improve convenience in diagnosis and reduce transmission.

Keywords: Automated diagnosis; Deep learning; Pharyngitis; Smartphone; Telemedicine; Tonsillitis.

MeSH terms

  • COVID-19
  • Coronavirus Infections
  • Deep Learning*
  • Humans
  • Image Interpretation, Computer-Assisted / methods*
  • Pandemics
  • Pharyngitis / diagnostic imaging*
  • Pharynx / diagnostic imaging
  • Photography
  • Pneumonia, Viral
  • Smartphone*
  • Telemedicine / methods*