Enhanced Image-Based Endoscopic Pathological Site Classification Using an Ensemble of Deep Learning Models

Abstract

In vivo diseases such as colorectal cancer and gastric cancer are increasingly occurring in humans. These are two of the most common types of cancer that cause death worldwide. Therefore, the early detection and treatment of these types of cancer are crucial for saving lives. With the advances in technology and image processing techniques, computer-aided diagnosis (CAD) systems have been developed and applied in several medical systems to assist doctors in diagnosing diseases using imaging technology. In this study, we propose a CAD method to preclassify the in vivo endoscopic images into negative (images without evidence of a disease) and positive (images that possibly include pathological sites such as a polyp or suspected regions including complex vascular information) cases. The goal of our study is to assist doctors to focus on the positive frames of endoscopic sequence rather than the negative frames. Consequently, we can help in enhancing the performance and mitigating the efforts of doctors in the diagnosis procedure. Although previous studies were conducted to solve this problem, they were mostly based on a single classification model, thus limiting the classification performance. Thus, we propose the use of multiple classification models based on ensemble learning techniques to enhance the performance of pathological site classification. Through experiments with an open database, we confirmed that the ensemble of multiple deep learning-based models with different network architectures is more efficient for enhancing the performance of pathological site classification using a CAD system as compared to the state-of-the-art methods.

Keywords: artificial intelligence; computer-aided diagnosis; ensemble learning; in vivo endoscopy; pathological site classification.

Figure 1

Overview of the proposed method.

Figure 2

(a) Example of a captured gastrointestinal endoscopic image and (b) the corresponding result of the preprocessing step.

Figure 3

Conventional architecture of convolutional neural networks.

Figure 4

Naïve inception architecture for rich feature extraction used in inception network.

Figure 5

Comparison between (a) plain convolutional blocks and (b) dense-connection block architecture.

Figure 6

Example of images: (a) Gastric images without the existence of pathological sites and (b) gastric images with the existence of pathological sites (Upper and lower images indicate those including polyp and complex vascular regions, respectively).

Figure 7

Training results (loss vs. accuracy) of Visual Geometry Group (VGG)-based, Inception-based, and DenseNet-based models in our experiments.

Figure 8

Receiver operating characteristic (ROC) curves of various system configurations in our experiment.

Figure 9

Examples of correct classification results by the proposed method: (a) True-negative cases and (b,c) true-positive cases (Upper and lower images indicate the ground-truth and testing images, respectively).

Figure 10

Examples of incorrect classification results by the proposed method: (a) False-positive cases and (b,c) false-negative cases (Upper and lower images indicate the ground-truth and testing images, respectively).

Figure 11

Obtained class activation maps corresponding to the pathological site class using (a) VGG-based network, (b) inception-based network, and (c) DenseNet-based network.