Progressive Rain Removal Based on the Combination Network of CNN and Transformer

Abstract

The rain removal method based on CNN develops rapidly. However, convolution operation has the disadvantages of limited receptive field and inadaptability to the input content. Recently, another neural network structure Transformer has shown excellent performance in natural language processing and advanced visual tasks by modeling global relationships, but Transformer has limitations in capturing local dependencies. To address the above limitations, we propose the combination network of CNN and Transformer, which fully combines the advantages of CNN and Transformer structure to complete the task of image restoration. We use CNN to provide preliminary output and adopt Transformer architecture to further optimize the output of CNN. In addition, by using some key designs in module connection, our model strengthens feature propagation and encourages feature reuse, allowing better information and gradient flow. The experimental results show that compared with the existing methods, our method can remove the rain lines more comprehensively and achieve the state-of-the-art results. Besides, the experimental results also demonstrate that the CNN structure can be effectively combined with Transformer to fully utilize the superiority of different structures.