Uncertainty-Driven Multi-Scale Feature Fusion Network for Real-time Image Deraining

Ming Tong,Xuefeng Yan,Yongzhen Wang

2023-07-19

Abstract:Visual-based measurement systems are frequently affected by rainy weather due to the degradation caused by rain streaks in captured images, and existing imaging devices struggle to address this issue in real-time. While most efforts leverage deep networks for image deraining and have made progress, their large parameter sizes hinder deployment on resource-constrained devices. Additionally, these data-driven models often produce deterministic results, without considering their inherent epistemic uncertainty, which can lead to undesired reconstruction errors. Well-calibrated uncertainty can help alleviate prediction errors and assist measurement devices in mitigating risks and improving usability. Therefore, we propose an Uncertainty-Driven Multi-Scale Feature Fusion Network (UMFFNet) that learns the probability mapping distribution between paired images to estimate uncertainty. Specifically, we introduce an uncertainty feature fusion block (UFFB) that utilizes uncertainty information to dynamically enhance acquired features and focus on blurry regions obscured by rain streaks, reducing prediction errors. In addition, to further boost the performance of UMFFNet, we fused feature information from multiple scales to guide the network for efficient collaborative rain removal. Extensive experiments demonstrate that UMFFNet achieves significant performance improvements with few parameters, surpassing state-of-the-art image deraining methods.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

The paper aims to address the problem of image deraining under rainy conditions, particularly to improve the performance of visual measurement systems in real-time applications. Specifically, the paper proposes a new architecture called the Uncertainty-driven Multi-scale Feature Fusion Network (UMFFNet) to tackle the following issues:

1. **Real-time and Resource Constraints**: Existing imaging devices face challenges in real-time processing of rainy images, especially deep learning models with a large number of parameters that are difficult to deploy on resource-constrained devices.
2. **Errors from Deterministic Results**: Existing data-driven models usually produce deterministic results, ignoring their inherent cognitive uncertainty, leading to reconstruction errors.
3. **Improving Model Performance with Uncertainty**: By introducing uncertainty estimation to guide the model to focus on areas obscured by rain, thereby reducing prediction errors and enhancing the model's pixel representation capability in uncertain regions.

UMFFNet achieves these goals through the following aspects:
- Designed an Uncertainty Feature Fusion Block (UFFB) that dynamically enhances the acquired features using uncertainty information.
- Combined a multi-scale architecture to fully exploit the similarity of rain streaks at different scales and effectively remove multi-scale rain streaks.
- Achieved significant performance improvement while ensuring a small number of model parameters, outperforming current state-of-the-art image deraining methods.

In summary, this research aims to develop a model that can efficiently run on resource-constrained devices and improve deraining performance through uncertainty estimation.