Abstract:Score-based diffusion methods provide a powerful strategy to solve image restoration tasks by flexibly combining a pre-trained foundational prior model with a likelihood function specified during test time. Such methods are predominantly derived from two stochastic processes: reversing Ornstein-Uhlenbeck, which underpins the celebrated denoising diffusion probabilistic models (DDPM) and denoising diffusion implicit models (DDIM), and the Langevin diffusion process. The solutions delivered by DDPM and DDIM are often remarkably realistic, but they are not always consistent with measurements because of likelihood intractability issues and the associated required approximations. Alternatively, using a Langevin process circumvents the intractable likelihood issue, but usually leads to restoration results of inferior quality and longer computing times. This paper presents a novel and highly computationally efficient image restoration method that carefully embeds a foundational DDPM denoiser within an empirical Bayesian Langevin algorithm, which jointly calibrates key model hyper-parameters as it estimates the model's posterior mean. Extensive experimental results on three canonical tasks (image deblurring, super-resolution, and inpainting) demonstrate that the proposed approach improves on state-of-the-art strategies both in image estimation accuracy and computing time.

What problem does this paper attempt to address?

This paper attempts to solve several key problems in image restoration tasks: 1. **The intractable likelihood function problem**: Many image restoration methods based on diffusion models need to deal with a computationally intractable likelihood function when using pre - trained diffusion models as implicit priors. This leads to the problem that the restoration results are inconsistent with the actual observed data. 2. **The trade - off between restoration quality and computational efficiency**: Existing methods either have high restoration quality but long computation time, or have short computation time but poor restoration quality. For example, methods using the Langevin process can avoid the intractable likelihood function problem, but usually result in lower - quality restoration results and longer computation time. 3. **Automatic adjustment of model parameters**: Existing image restoration methods often require manual adjustment of the regularization parameters of the model, which is not only time - consuming but may also lead to sub - optimal results. To address these problems, the paper proposes a new image restoration method, which achieves efficient and high - quality image restoration by embedding a pre - trained denoising diffusion probability model (DDPM) into the empirical Bayes Langevin algorithm. Specifically, this method has the following features: - **Direct use of the original likelihood function**: By embedding the pre - trained DDPM denoiser, this method can directly use the original likelihood function, thus avoiding the intractable likelihood function problem. - **Automatic adjustment of model parameters**: Using empirical Bayes techniques, this method can automatically adjust the regularization parameters of the model, improving the robustness and adaptability of the model. - **Efficient computational performance**: By implementing plug - and - play Langevin sampling (PnP - ULA) in the latent space, this method significantly accelerates the convergence speed and improves the accuracy of the restoration results. Experimental results show that this method is not only superior to existing methods in terms of image estimation accuracy, but also competitive in computation time in tasks such as image deblurring, super - resolution, and inpainting.

Empirical Bayesian image restoration by Langevin sampling with a denoising diffusion implicit prior

Regularization by denoising: Bayesian model and Langevin-within-split Gibbs sampling

Diffusion Priors for Variational Likelihood Estimation and Image Denoising

Frequency-Guided Posterior Sampling for Diffusion-Based Image Restoration

Diffusion Posterior Proximal Sampling for Image Restoration

A stochastic optimisation unadjusted Langevin method for empirical Bayesian estimation in semi-blind image deblurring problems

Image Restoration by Denoising Diffusion Models with Iteratively Preconditioned Guidance

Effective Image Restorations Using a Novel Spatial Adaptive Prior

Decoupled Data Consistency with Diffusion Purification for Image Restoration

Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstruction

Bayesian image restoration using a large-scale total patch variation prior

Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise

Stochastic Deep Restoration Priors for Imaging Inverse Problems

Generative Diffusion Prior for Unified Image Restoration and Enhancement

Posterior sampling via Langevin dynamics based on generative priors

Robust image restoration via adaptive low-rank approximation and joint kernel regression.

Restoration by Generation with Constrained Priors

Improving Diffusion-Based Image Restoration with Error Contraction and Error Correction

Image Restoration via Simultaneous Sparse Coding and Gaussian Scale Mixture

Accelerated Bayesian Imaging by Relaxed Proximal-Point Langevin Sampling