Abstract:Recent diffusion models provide a promising zero-shot solution to noisy linear inverse problems without retraining for specific inverse problems. In this paper, we reveal that recent methods can be uniformly interpreted as employing a Gaussian approximation with hand-crafted isotropic covariance for the intractable denoising posterior to approximate the conditional posterior mean. Inspired by this finding, we propose to improve recent methods by using more principled covariance determined by maximum likelihood estimation. To achieve posterior covariance optimization without retraining, we provide general plug-and-play solutions based on two approaches specifically designed for leveraging pre-trained models with and without reverse covariance. We further propose a scalable method for learning posterior covariance prediction based on representation with orthonormal basis. Experimental results demonstrate that the proposed methods significantly enhance reconstruction performance without requiring hyperparameter tuning.

What problem does this paper attempt to address?

This paper attempts to address the problem of how to improve existing diffusion model methods to enhance the performance of conditional sampling when dealing with noisy linear inverse problems. Specifically, the paper focuses on zero-shot methods, which utilize pre-trained unconditional diffusion models for conditional sampling in various inverse problem scenarios without retraining the model. ### Background of the Paper Noisy linear inverse problems are common in image processing, including tasks such as denoising, inpainting, deblurring, and super-resolution. These problems are typically modeled as linear projections of images under noise interference. In recent years, diffusion models have gained widespread attention for their potential in solving inverse problems. Depending on the training strategy, diffusion models can be divided into two categories: 1. **Supervised Methods**: Learning conditional diffusion models through paired datasets of degraded and clean images. 2. **Zero-Shot Methods**: Utilizing pre-trained unconditional diffusion models for conditional sampling in various inverse problem scenarios without retraining. ### Existing Problems Although existing zero-shot methods are effective to some extent, they have shortcomings in optimizing the consistency of conditional sampling. These methods typically employ techniques such as projecting samples into the measurement subspace, modifying the sampling process using classifier guidance, or using variational inference to ensure the consistency of conditional sampling. However, these methods have limitations in approximating the conditional posterior mean \( E[x_0 | x_t, y] \). ### Contributions of the Paper 1. **Unified Interpretation**: The paper reveals that recent zero-shot methods can be uniformly interpreted as using isotropic Gaussian approximations to approximate the infeasible denoising posterior \( p(x_0 | x_t) \), thereby approximating the conditional posterior mean \( E[x_0 | x_t, y] \). 2. **Improved Method**: Based on maximum likelihood estimation (MLE) to optimize the posterior covariance, a general plug-in solution is proposed that can achieve posterior covariance optimization without retraining. 3. **Scalable Method**: To overcome the quadratic complexity of covariance prediction, a scalable method based on orthogonal basis is proposed for learning posterior covariance prediction. ### Experimental Results Experimental results show that the proposed method significantly improves reconstruction performance in various tasks (such as inpainting, deblurring, and super-resolution) without the need for hyperparameter tuning. ### Conclusion By optimizing the posterior covariance, the paper improves existing diffusion model methods, making them perform better in handling noisy linear inverse problems, with higher robustness and flexibility.

Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance

Improving Probabilistic Diffusion Models With Optimal Covariance Matching

Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems

Gaussian is All You Need: A Unified Framework for Solving Inverse Problems via Diffusion Posterior Sampling

Tweedie Moment Projected Diffusions For Inverse Problems

A Variational Perspective on Solving Inverse Problems with Diffusion Models

Diffusion Posterior Sampling for General Noisy Inverse Problems

Removing Structured Noise with Diffusion Models

Diffusion Model Based Posterior Sampling for Noisy Linear Inverse Problems

Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing

Enhancing Diffusion Posterior Sampling for Inverse Problems by Integrating Crafted Measurements

Interpreting and Improving Diffusion Models from an Optimization Perspective

Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision

Think Twice Before You Act: Improving Inverse Problem Solving With MCMC

Diffusion Prior-Based Amortized Variational Inference for Noisy Inverse Problems

Constrained Diffusion Implicit Models

Improving Decoupled Posterior Sampling for Inverse Problems using Data Consistency Constraint

Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs

Diffusion Priors for Variational Likelihood Estimation and Image Denoising

Accelerating Diffusion Models for Inverse Problems through Shortcut Sampling