Abstract:Supervised image denoising methods based on deep neural networks require a large amount of noisy-clean or noisy image pairs for network training. Thus, their performance drops drastically when the given noisy image is significantly different from the training data. Recently, several unsupervised learning models have been proposed to reduce the dependence on training data. Although unsupervised methods only require noisy images for learning, their denoising effect is relatively weak compared with supervised methods. This paper proposes a two-stage unsupervised deep learning framework based on deep image prior (DIP) to enhance the image denoising performance. First, a two-target DIP learning strategy is proposed to impose a learning restriction on the DIP optimization process. A cleaner preliminary image, together with the given noisy image, was used as the learning target of the two-target DIP learning process. We then demonstrate that adding an extra learning target with better image quality in the DIP learning process is capable of constraining the search space of the optimization process and improving the denoising performance. Furthermore, we observe that given the same network input and the same learning target, the DIP optimization process cannot generate the same denoised images. This indicates that the denoised results are uncertain, although they are similar in image quality and are complemented by local details. To utilize the uncertainty of the DIP, we employ a supervised denoising method to preprocess the given noisy image and propose an up- and down-sampling strategy to produce multiple sampled instances of the preprocessed image. These sampled instances were then fed into multiple two-target DIP learning processes to generate multiple denoised instances with different image details. Finally, we propose an unsupervised fusion network that fuses multiple denoised instances into one denoised image to further improve the denoising effect. We evaluated the proposed method through extensive experiments, including grayscale image denoising, color image denoising, and real-world image denoising. The experimental results demonstrate that the proposed framework outperforms unsupervised methods in all cases, and the denoising performance of the framework is close to or superior to that of supervised denoising methods for synthetic noisy image denoising and significantly outperforms supervised denoising methods for real-world image denoising. In summary, the proposed method is essentially a hybrid method that combines both supervised and unsupervised learning to improve denoising performance. Adopting a supervised method to generate preprocessed denoised images can utilize the external prior and help constrict the search space of the DIP, whereas using an unsupervised method to produce intermediate denoised instances can utilize the internal prior and provide adaptability to various noisy images of a real scene.

Enhancing Deep Image Prior with Roughly Clean Pairs and Spatially Random Sampling

Boosting Deep Image Prior by Integrating External and Internal Image Priors

Multi-channel deep image prior for image denoising

An Improved DIP Denoising Model Based on Internal and External Image Priors and Image Fusion

A deep image prior-based three-stage denoising method using generative and fusion strategies

Learning from Multiple Instances: A Two-Stage Unsupervised Image Denoising Framework Based on Deep Image Prior

Image Denoising Using a Novel Deep Generative Network with Multiple Target Images and Adaptive Termination Condition

Dual-branch Deep Image Prior for Image Denoising.

Mixed Degradation Image Restoration Via Deep Image Prior Empowered by Deep Denoising Engine

Learning Multi‐Scale Deep Image Prior for High‐Quality Unsupervised Image Denoising

A Triple Deep Image Prior Model for Image Denoising Based on Mixed Priors and Noise Learning

Boosting Image Denoising Effect Via Low-Level Noise Injection

Hyperspectral Mixed Noise Removal Via Spatial-Spectral Constrained Unsupervised Deep Image Prior

Hyperspectral Image Denoising Via Double Subspace Deep Prior

Unsupervised Hyperspectral Mixed Noise Removal Via Spatial-Spectral Constrained Deep Image Prior

Dilated Residual Encode-Decode Networks for Image Denoising

Hyperspectral Denoising Using Unsupervised Disentangled Spatio-Spectral Deep Priors

A Masked-Pre-Training-Based Fast Deep Image Prior Denoising Model

Hyperspectral Denoising Using Unsupervised Disentangled Spatiospectral Deep Priors

An Adaptive Image Denoising Method Based on Deep Rectified Denoising Auto-Encoder

Deep Marginalized Sparse Denoising Auto-Encoder for Image Denoising