Boosting Deep Image Prior by Integrating External and Internal Image Priors

Shaoping Xu,Xiaohui Cheng,Jie Luo,Xiaojun Chen,Nan Xiao
DOI: https://doi.org/10.1117/1.jei.32.1.013021
IF: 0.829
2023-01-01
Journal of Electronic Imaging
Abstract:Recently, an unsupervised method [denoted as deep image prior (DIP)] that interprets images as the output of a convolutional network with randomly initialized inputs has received much attention, as the DIP can adapt its network parameters to any given noisy image without requiring training pairs, providing a great flexibility for handling those noisy images with complex structures and low degree of self-similarity. However, the DIP method still suffers from a crucial limitation that only the noisy image is used as target image, which implies that only internal prior information is used for noise removal and leads to poor denoising performance, compared with supervised denoising models. In this work, we aim to boost the DIP method by integrating external and internal image priors. Specifically, with a given noisy image, we first exploited a state-of-the-art supervised method to denoise it and obtain its corresponding denoised image called initial denoised image. Then the initial denoised image containing rich external prior information was used as extra target image together with the given noisy image within the standard DIP framework, resulting in a highly effective unsupervised recovery process. Next, to take full advantage of the uncertainty of the DIP network, the above DIP denoising routine with the different random inputs was executed multiple times to generate enough complementary denoised images (samples). Finally, with an unsupervised weight map generative network, the generated samples were fused in a pixel-wise manner. The fused image with better image quality was treated as the final denoised image. We had verified the denoising performance of our method on a large of images from several benchmark databases as well as real-world noisy images. The comparative results show that our method outperforms the original DIP method and other unsupervised methods by a large margin, and surpasses state-of-the-art supervised counterparts with comparable peak signal-to-noise rate.
What problem does this paper attempt to address?