A deep-learning based multimodal system for Covid-19 diagnosis using breathing sounds and chest X-ray images

Unais Sait; Gokul Lal K V; Sanjana Shivakumar; Tarun Kumar; Rahul Bhaumik; Sunny Prajapati; Kriti Bhalla; Anaghaa Chakrapani

doi:10.1016/j.asoc.2021.107522

A deep-learning based multimodal system for Covid-19 diagnosis using breathing sounds and chest X-ray images

Appl Soft Comput. 2021 Sep:109:107522. doi: 10.1016/j.asoc.2021.107522. Epub 2021 May 26.

Authors

Unais Sait¹, Gokul Lal K V², Sanjana Shivakumar³, Tarun Kumar⁴, Rahul Bhaumik¹, Sunny Prajapati¹, Kriti Bhalla⁵, Anaghaa Chakrapani⁶

Affiliations

¹ Faculty of Architecture and Design, PES University, Bengaluru, India.
² East Point College of Engineering and Technology, Bengaluru, India.
³ Department of Design and Computation Arts, Concordia University, Qc, Canada.
⁴ Centre for Product Design and Manufacturing, Indian Institute of Science, Bengaluru, India.
⁵ School of Architecture, Ramaiah Institute of Technology, Bengaluru, Karnataka, India.
⁶ School of Design, Avantika University, Ujjain, India.

Abstract

Covid-19 has become a deadly pandemic claiming more than three million lives worldwide. SARS-CoV-2 causes distinct pathomorphological alterations in the respiratory system, thereby acting as a biomarker to aid its diagnosis. A multimodal framework (Ai-CovScan) for Covid-19 detection using breathing sounds, chest X-ray (CXR) images, and rapid antigen test (RAnT) is proposed. Transfer Learning approach using existing deep-learning Convolutional Neural Network (CNN) based on Inception-v3 is combined with Multi-Layered Perceptron (MLP) to develop the CovScanNet model for reducing false-negatives. This model reports a preliminary accuracy of 80% for the breathing sound analysis, and 99.66% Covid-19 detection accuracy for the curated CXR image dataset. Based on Ai-CovScan, a smartphone app is conceptualised as a mass-deployable screening tool, which could alter the course of this pandemic. This app's deployment could minimise the number of people accessing the limited and expensive confirmatory tests, thereby reducing the burden on the severely stressed healthcare infrastructure.

Keywords: Breathing sounds; CNN; Chest X-ray images; Covid-19; Deep-learning; MLP.