A Survey on Rain Removal from Video and Single Image

Hong Wang,Yichen Wu,Minghan Li,Qian Zhao,Deyu Meng

DOI: https://doi.org/10.1007/s11432-020-3225-9

2019-10-04

Abstract:Rain streaks might severely degenerate the performance of video/image processing tasks. The investigations on rain removal from video or a single image has thus been attracting much research attention in the field of computer vision and pattern recognition, and various methods have been proposed against this task in the recent years. However, there is still not a comprehensive survey paper to summarize current rain removal methods and fairly compare their generalization performance, and especially, still not a off-the-shelf toolkit to accumulate recent representative methods for easy performance comparison and capability evaluation. Aiming at this meaningful task, in this study we present a comprehensive review for current rain removal methods for video and a single image. Specifically, these methods are categorized into model-driven and data-driven approaches, and more elaborate branches of each approach are further introduced. Intrinsic capabilities, especially generalization, of representative state-of-the-art methods of each approach have been evaluated and analyzed by experiments implemented on synthetic and real data both visually and quantitatively. Furthermore, we release a comprehensive repository, including direct links to 74 rain removal papers, source codes of 9 methods for video rain removal and 20 ones for single image rain removal, 19 related project pages, 6 synthetic datasets and 4 real ones, and 4 commonly used image quality metrics, to facilitate reproduction and performance comparison of current existing methods for general users. Some limitations and research issues worthy to be further investigated have also been discussed for future research of this direction.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the serious impact of raindrops or rain streaks on the performance of video and image processing tasks. Specifically, rain streaks may severely degrade the image quality in computer vision and pattern recognition tasks, such as object tracking, scene analysis, person re - identification, and event detection. Therefore, removing rain streaks from videos or single images has become a necessary pre - processing step for these subsequent tasks to further improve their performance.

The paper points out that although various rain streak removal methods have been proposed in recent years, there is still a lack of a comprehensive review paper to summarize the current rain streak removal methods and fairly compare their generalization performance. In addition, there is no ready - made toolkit available that can bring together recent representative methods for users to easily perform performance comparison and capability evaluation. For this meaningful task, the authors provide a comprehensive review of current video and single - image rain streak removal methods in this study, specifically including the following points:

1. **Method Classification**: The existing rain streak removal methods are divided into two major categories: model - based methods and data - based methods, and the main techniques and representative methods of each method are further introduced in detail.
2. **Performance Evaluation**: Through experiments on synthetic and real - world datasets, the intrinsic capabilities, especially the generalization capabilities, of the most representative state - of - the - art methods are evaluated both visually and quantitatively.
3. **Resource Release**: A comprehensive resource library is released, which contains links to 74 rain streak removal papers, source codes of 9 video rain streak removal methods and 20 single - image rain streak removal methods, 19 related project pages, 6 synthetic datasets and 4 real - world datasets, and 4 commonly used image quality metrics, to facilitate users to reproduce and compare the performance of existing methods.

Overall, this paper aims to promote the research progress in the field of rain streak removal through systematic review and evaluation, and provide an easy - to - use platform for researchers and engineers to better understand and apply the existing rain streak removal techniques.