Reconciling Hand-Crafted and Self-Supervised Deep Priors for Video Directional Rain Streaks Removal

Jun-Hao Zhuang,Yi-Si Luo,Xi-Le Zhao,Tai-Xiang Jiang

DOI: https://doi.org/10.1109/lsp.2021.3120598

2021-01-01

IEEE Signal Processing Letters

Abstract:Removing rain streaks in videos has recently received much attention. Existing hand-crafted priors-based methods suffer from limited representation abilities, and supervised deep learning methods need high-quality training data. This paper proposes a novel video rain streaks removal method by reconciling hand-crafted and self-supervised deep priors. The hand-crafted priors include the learned gradient prior, the sparse prior, and the temporal local smooth prior. Meanwhile, a deep convolutional neural network is employed to self-supervisedly capture the deep prior of the clean video without any training data. Our method organically integrates hand-crated priors and self-supervised deep priors to achieve both high generalization abilities and representation abilities. Thus, our method can faithfully remove directional rain streaks in real world videos. To address the resulting model, we introduce an alternating direction multiplier method algorithm. Extensive experiments validate the superiority of our method over state-of-the-art methods.