Abstract:Diffusion models have become increasingly popular for generative modeling due to their ability to generate high-quality samples. This has unlocked exciting new possibilities for solving inverse problems, especially in image restoration and reconstruction, by treating diffusion models as unsupervised priors. This survey provides a comprehensive overview of methods that utilize pre-trained diffusion models to solve inverse problems without requiring further training. We introduce taxonomies to categorize these methods based on both the problems they address and the techniques they employ. We analyze the connections between different approaches, offering insights into their practical implementation and highlighting important considerations. We further discuss specific challenges and potential solutions associated with using latent diffusion models for inverse problems. This work aims to be a valuable resource for those interested in learning about the intersection of diffusion models and inverse problems.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is **how to use pre - trained diffusion models to solve inverse problems without further training**. Specifically, the paper focuses on how to use diffusion models as unsupervised priors to solve inverse problems in areas such as image restoration and reconstruction. Inverse problems usually involve recovering the original signal or image from observed data, and such problems are widely used in different scientific fields, such as seismic imaging, weather prediction, oceanography, audio signal processing, medical imaging, etc. ### Problem Background The mathematical setting of inverse problems is usually that the measured value \(y\in\mathbb{R}^m\) and a noise model \(Y = A(X)+\sigma_yZ\) are known, where \(Z\sim N(0, I_m)\), and the goal is to recover the unknown sample \(x\in\mathbb{R}^n\) from these measured values. Common inverse problems include denoising, inpainting, compressed sensing, convolution, etc. ### Core Problems of the Paper 1. **Using Pre - trained Diffusion Models**: The paper explores how to use already - trained diffusion models to solve inverse problems without retraining for each new inverse problem. Diffusion models have become increasingly popular in generative modeling because of their ability to generate high - quality samples, which provides new possibilities for solving inverse problems. 2. **Unsupervised Methods**: The paper focuses on unsupervised methods, that is, methods that focus on characterizing the target signal distribution \(p_X\) and applying this knowledge in the inverse process. These methods do not depend on paired data and can therefore be flexibly applied to different inverse problems. 3. **Method Classification**: The paper classifies existing methods into four main categories: - **Explicit Approximate Measurement Matching**: These methods approximate the measurement - matching score \(\nabla\log p_t(y|x_t)\) through a closed - form expression. - **Variational Inference**: These methods approximate the true posterior distribution \(p(x|y)\) with simpler, tractable distributions and use variational formulas to optimize the parameters of these simple distributions. - **CSGM - type Methods**: These methods change the initial noise of the deterministic diffusion sampler through back - propagation, essentially optimizing in the latent space of the diffusion model. - **Asymptotically Exact Methods**: These methods aim to sample from the true posterior distribution, usually by constructing a Markov chain (MCMC) or propagating particles through a series of distributions (SMC) to obtain samples of the approximate posterior. ### Specific Challenges and Solutions - **Non - linear Problems**: The paper discusses how to handle non - linear inverse problems, that is, when \(A\) is a non - linear operator. - **Blind Processing**: The paper also deals with the blind - processing problem, that is, the case where the forward model is unknown. - **Noise Handling**: How to effectively use diffusion models in the presence of noise. - **Pixel/Latent - space Operations**: Differences in the operations of different methods in the pixel space or the latent space. - **Text Conditions**: Some methods support solving inverse problems under text conditions. - **Optimization Techniques**: Types of optimization techniques used, such as gradient descent, projection, etc. ### Conclusion The paper aims to provide a valuable resource for readers interested in the research at the intersection of diffusion models and inverse problems. By systematically reviewing and classifying existing methods, it helps readers understand the connections and unified principles among these methods.

A Survey on Diffusion Models for Inverse Problems

Diffusion Models: A Comprehensive Survey of Methods and Applications

Diffusion models in medical imaging: A comprehensive survey

Diffusion Models in Vision: A Survey

Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision

Diffusion Models for Medical Image Analysis: A Comprehensive Survey

A Comprehensive Survey on Diffusion Models and Their Applications

Inverse Problems with Diffusion Models: A MAP Estimation Perspective

Diffusion Models for Image Restoration and Enhancement - A Comprehensive Survey

Tweedie Moment Projected Diffusions For Inverse Problems

A Variational Perspective on Solving Inverse Problems with Diffusion Models

Diffusion Posterior Sampling for General Noisy Inverse Problems

Diffusion Models in Low-Level Vision: A Survey

Improving Diffusion Models for Inverse Problems using Manifold Constraints

Diffusion State-Guided Projected Gradient for Inverse Problems

Enhancing Diffusion Posterior Sampling for Inverse Problems by Integrating Crafted Measurements

Removing Structured Noise with Diffusion Models

Diffusion Models, Image Super-Resolution And Everything: A Survey