Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis

Kai Zhang,Yawei Li,Jingyun Liang,Jiezhang Cao,Yulun Zhang,Hao Tang,Deng-Ping Fan,Radu Timofte,Luc Van Gool

DOI: https://doi.org/10.1007/s11633-023-1466-0

2023-12-01

Abstract:While recent years have witnessed a dramatic upsurge of exploiting deep neural networks toward solving image denoising, existing methods mostly rely on simple noise assumptions, such as additive white Gaussian noise (AWGN), JPEG compression noise and camera sensor noise, and a general-purpose blind denoising method for real images remains unsolved. In this paper, we attempt to solve this problem from the perspective of network architecture design and training data synthesis. Specifically, for the network architecture design, we propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block, and then plug it as the main building block into the widely-used image-to-image translation UNet architecture. For the training data synthesis, we design a practical noise degradation model which takes into consideration different kinds of noise (including Gaussian, Poisson, speckle, JPEG compression, and processed camera sensor noises) and resizing, and also involves a random shuffle strategy and a double degradation strategy. Extensive experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance and the new degradation model can help to significantly improve the practicability. We believe our work can provide useful insights into current denoising research.

Computer Vision and Pattern Recognition,Graphics,Image and Video Processing

What problem does this paper attempt to address?

The paper attempts to address the issue of the lack of a universal blind denoising method in practical image denoising. Most existing methods rely on simple noise assumptions (such as additive white Gaussian noise (AWGN), JPEG compression noise, and camera sensor noise), but these methods perform poorly when dealing with complex noise in the real world. The authors of the paper tackle this problem from two aspects: network architecture design and training data synthesis, proposing a new method to address this issue.

Specifically:
1. **Network Architecture Design**: The Swin-Conv block is proposed, which combines the local modeling capability of residual convolution layers and the non-local modeling capability of Swin Transformer blocks, and inserts it as the main building block into the commonly used UNet architecture.
2. **Training Data Synthesis**: A practical noise degradation model is designed, considering multiple types of noise (including Gaussian noise, Poisson noise, speckle noise, JPEG compression noise, and post-processing camera sensor noise) as well as resampling operations, and employing a random shuffling strategy and a dual degradation strategy.