Abstract:The recent emergence of diffusion models has significantly advanced the precision of learnable priors, presenting innovative avenues for addressing inverse problems. Since inverse problems inherently entail maximum a posteriori estimation, previous works have endeavored to integrate diffusion priors into the optimization frameworks. However, prevailing optimization-based inverse algorithms primarily exploit the prior information within the diffusion models while neglecting their denoising capability. To bridge this gap, this work leverages the diffusion process to reframe noisy inverse problems as a two-variable constrained optimization task by introducing an auxiliary optimization variable. By employing gradient truncation, the projection gradient descent method is efficiently utilized to solve the corresponding optimization problem. The proposed algorithm, termed ProjDiff, effectively harnesses the prior information and the denoising capability of a pre-trained diffusion model within the optimization framework. Extensive experiments on the image restoration tasks and source separation and partial generation tasks demonstrate that ProjDiff exhibits superior performance across various linear and nonlinear inverse problems, highlighting its potential for practical applications. Code is available at <a class="link-external link-https" href="https://github.com/weigerzan/ProjDiff/" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

This paper mainly discusses how to fully utilize the denoising capability of diffusion models to solve inverse problems. Inverse problems are optimization problems widely found in image restoration, medical image processing, 3D vision, audio processing, and other fields. Existing methods mostly attempt to use diffusion models as prior information in the optimization framework, but overlook their potential in denoising. The paper proposes a new algorithm called "ProjDiff", which transforms the noisy inverse problem into a two-variable constrained optimization task by introducing an auxiliary optimization variable. By utilizing gradient truncation technique, the algorithm effectively solves the corresponding optimization problem, thus simultaneously leveraging the prior information and denoising capability in pretrained diffusion models. ProjDiff is not only applicable to linear inverse problems, but also extends to nonlinear problems, demonstrating its superior performance on various inverse problems. The experimental section demonstrates the outstanding performance of ProjDiff in image restoration tasks, source separation, and partial generation tasks. It outperforms existing state-of-the-art methods. Additionally, ProjDiff addresses the issue of weak observations by adjusting the noise schedule to adapt to different observation intensities. In conclusion, this paper aims to address how to more effectively combine the prior knowledge and denoising capability of diffusion models. It proposes a new optimization approach that has important theoretical and practical value for solving inverse problems.

Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems

Diffusion Priors for Variational Likelihood Estimation and Image Denoising

Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision

A Variational Perspective on Solving Inverse Problems with Diffusion Models

Interpreting and Improving Diffusion Models from an Optimization Perspective

Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing

Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance

Diffusion Prior-Based Amortized Variational Inference for Noisy Inverse Problems

Stimulating the Diffusion Model for Image Denoising Via Adaptive Embedding and Ensembling

Stimulating Diffusion Model for Image Denoising via Adaptive Embedding and Ensembling

Enhancing Diffusion Posterior Sampling for Inverse Problems by Integrating Crafted Measurements

Diffusion State-Guided Projected Gradient for Inverse Problems

DDGM: Solving inverse problems by Diffusive Denoising of Gradient-based Minimization

Denoising Diffusion Models for Plug-and-Play Image Restoration

Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise

Tweedie Moment Projected Diffusions For Inverse Problems

Principled Probabilistic Imaging using Diffusion Models as Plug-and-Play Priors

Real-World Denoising via Diffusion Model

Learn to Optimize Denoising Scores: A Unified and Improved Diffusion Prior for 3D Generation