Recurrent residual U-Net for medical image segmentation

Abstract

Deep learning (DL)-based semantic segmentation methods have been providing state-of-the-art performance in the past few years. More specifically, these techniques have been successfully applied in medical image classification, segmentation, and detection tasks. One DL technique, U-Net, has become one of the most popular for these applications. We propose a recurrent U-Net model and a recurrent residual U-Net model, which are named RU-Net and R2U-Net, respectively. The proposed models utilize the power of U-Net, residual networks, and recurrent convolutional neural networks. There are several advantages to using these proposed architectures for segmentation tasks. First, a residual unit helps when training deep architectures. Second, feature accumulation with recurrent residual convolutional layers ensures better feature representation for segmentation tasks. Third, it allows us to design better U-Net architectures with the same number of network parameters with better performance for medical image segmentation. The proposed models are tested on three benchmark datasets, such as blood vessel segmentation in retinal images, skin cancer segmentation, and lung lesion segmentation. The experimental results show superior performance on segmentation tasks compared to equivalent models, including a variant of a fully connected convolutional neural network called SegNet, U-Net, and residual U-Net.

Keywords: U-Net; convolutional neural networks; medical imaging; recurrent U-Net; recurrent residual U-Net; residual U-Net; semantic segmentation.

Fig. 1

Medical image segmentation examples displaying RBVS on the left, skin cancer lesion segmentation in the middle, and LS on the right.

Fig. 2

The RU-Net architecture with convolutional encoding and decoding units using RCLs, which is based on a U-Net architecture. The residual units are used with the RCL for R2U-Net architectures.

Fig. 3

Different variants of the convolutional and recurrent convolutional units (RCUs) including (a) the forward convolutional unit, (b) the recurrent convolutional block, (c) the residual convolutional unit, and (d) the recurrent residual convolutional unit.

Fig. 4

The lower part of units represents RCUs and upper parts are for unfolded RCUs for $t=2$ (left) and $t=3$ (right). For $t=2$, we have used one forward convolutional layer followed by two RCLs; on the other hand, for $t=3$, one forward convolutional layer is used followed by three RCLs. The orange and blue arrows represent the equivalent representation of folded and unfolded RCUs and the convolutional operation with respect to different time steps, respectively. The orange and green rectangles indicate the kernels and the feature maps for the respective layers.

Fig. 5

Example images from training datasets where (a) is taken from the DRIVE dataset, (b) is taken from the STARE dataset, and (c) is taken from the CHASE-DB1 dataset. The first row shows the original images, the second row shows the FOVs, and third row shows the target outputs.

Fig. 6

Example patches are shown in (a) and the corresponding outputs of the patches are shown in (b).

Fig. 7

Training and validation AC of the proposed RU-Net and R2U-Net models compared to the ResU-Net and U-Net models for blood vessel segmentation task. (a) Training AC and (b) validation.

Fig. 8

Experimental outputs for three different datasets for RBVS using R2U-Net. The first row shows input images in grayscale, the second row shows the ground truth, and the third row shows the experimental outputs. The images correspond to the (a) DRIVE, (b) STARE, and (c) CHASE_DB1 datasets.

Fig. 9

AUC for RBVS for the best performance achieved with R2U-Net on three different datasets.

Fig. 10

Training and validation AC of R2U-Net, RU-Net, ResU-Net, and U-Net for SLS. (a) Training AC and (b) validation AC.

Fig. 11

Illustration of qualitative assessment of the proposed R2U-Net for the skin cancer segmentation task. (a) The input sample, (b) ground truth, (c) the outputs from the SegNet model, (d) the outputs from the U-Net model, and (e) the results of the proposed R2U-Net model.

Fig. 12

The experimental results for LS, where (a) shows the inputs, (b) shows the ground truth, (c) shows the outputs of SegNet, (d) shows the outputs of U-Net, and (e) shows the outputs of R2U-Net.

Fig. 13

ROC curve for LS for four different models, where $t=3$.

Fig. 14

The performance of the three different models (SegNet, U-Net, and R2U-Net) for different numbers of training and validation samples, where (a) the training DI coefficient errors (1-DI) and (b) validation DI coefficient errors for five different trials are displayed.

Fig. 15

Testing errors of the R2U-Net, SegNet, and U-Net models for different split ratios for the LS application.