Abstract:Due to the development of deep learning-based denoisers, the plug-and-play strategy has achieved great success in image restoration problems. However, existing plug-and-play image restoration methods are designed for non-blind Gaussian denoising such as zhang et al (2022), the performance of which visibly deteriorate for unknown noises. To push the limits of plug-and-play image restoration, we propose a novel framework with blind Gaussian prior, which can deal with more complicated image restoration problems in the real world. More specifically, we build up a new image restoration model by regarding the noise level as a variable, which is implemented by a two-stage blind Gaussian denoiser consisting of a noise estimation subnetwork and a denoising subnetwork, where the noise estimation subnetwork provides the noise level to the denoising subnetwork for blind noise removal. We also introduce the curvature map into the encoder-decoder architecture and the supervised attention module to achieve a highly flexible and effective convolutional neural network. The experimental results on image denoising, deblurring and single-image super-resolution are provided to demonstrate the advantages of our deep curvature denoiser and the resulting plug-and-play blind image restoration method over the state-of-the-art model-based and learning-based methods. Our model is shown to be able to recover the fine image details and tiny structures even when the noise level is unknown for different image restoration tasks. The source codes are available at https://github.com/Duanlab123/CurvPnP.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is dealing with the situation of unknown noise levels in image restoration. Existing Plug - and - Play (PnP) image restoration methods are mainly designed for non - blind Gaussian denoising, and their performance will decline significantly when facing unknown noise. To overcome this limitation, the paper proposes a new framework - PnP image restoration method with blind Gaussian prior (CurvPnP), which can handle more complex image restoration problems in the real world. Specifically, the paper constructs a new image restoration model by regarding the noise level as a variable. This model consists of a two - stage blind Gaussian denoiser, including a noise estimation sub - network and a denoising sub - network. The noise estimation sub - network provides the noise level to the denoising sub - network to achieve blind noise removal. In addition, the paper also introduces the curvature map into the encoder - decoder architecture and introduces the supervised attention module to achieve a highly flexible and effective convolutional neural network. The main contributions of the paper include: 1. Proposing an effective and powerful two - stage blind image restoration method, including a noise estimation sub - network and a denoising sub - network, called C - UNet. In particular, these two sub - networks are constructed by using ConvNeXt blocks and the encoder - decoder architecture to maintain efficiency and improve model performance. 2. Developing a new denoiser containing rich curvature information, where the Gaussian curvature map is not only used as a prior but also used in the supervised attention module to refine features. Thanks to the curvature information, this denoiser performs well in restoring image edges and fine structures. 3. Implementing an iterative PnP strategy for solving different image restoration problems, such as image deblurring and super - resolution. Experimental results show that CurvPnP has very competitive performance. The paper verifies the advantages of this method in image denoising, deblurring and single - image super - resolution tasks through experiments, especially in the case of unknown noise levels, it can restore the details and tiny structures of the image.

CurvPnP: Plug-and-play Blind Image Restoration with Deep Curvature Denoiser

Plug-and-Play Image Restoration with Deep Denoiser Prior

Multinoise-type Blind Denoising Using a Single Uniform Deep Convolutional Neural Network.

REPNP: Plug-and-Play with Deep Reinforcement Learning Prior for Robust Image Restoration

Denoising Diffusion Models for Plug-and-Play Image Restoration

Dilated Residual Encode-Decode Networks for Image Denoising

Deep Plug-and-Play Prior for Hyperspectral Image Restoration

Video Restoration with a Deep Plug-and-Play Prior

Image Restoration using Plug-and-Play CNN MAP Denoisers

Learning Deep CNN Denoiser Prior for Image Restoration

Constrained Plug-and-Play Priors for Image Restoration

Denoising Prior Driven Deep Neural Network for Image Restoration

3D Priors-Guided Diffusion for Blind Face Restoration

Cross-Patch Graph Convolutional Network for Image Denoising.

Deep Variational Network Toward Blind Image Restoration

Plug-and-Play image restoration with Stochastic deNOising REgularization

Deep Graph Laplacian Regularization for Robust Denoising of Real Images

Restoration by Generation with Constrained Priors

View Blind-spot as Inpainting: Self-Supervised Denoising with Mask Guided Residual Convolution

A new diffusion method for blind image denoising