Diffusion Posterior Sampling for General Noisy Inverse Problems

Hyungjin Chung,Jeongsol Kim,Michael T. Mccann,Marc L. Klasky,Jong Chul Ye

2024-05-20

Abstract:Diffusion models have been recently studied as powerful generative inverse problem solvers, owing to their high quality reconstructions and the ease of combining existing iterative solvers. However, most works focus on solving simple linear inverse problems in noiseless settings, which significantly under-represents the complexity of real-world problems. In this work, we extend diffusion solvers to efficiently handle general noisy (non)linear inverse problems via approximation of the posterior sampling. Interestingly, the resulting posterior sampling scheme is a blended version of diffusion sampling with the manifold constrained gradient without a strict measurement consistency projection step, yielding a more desirable generative path in noisy settings compared to the previous studies. Our method demonstrates that diffusion models can incorporate various measurement noise statistics such as Gaussian and Poisson, and also efficiently handle noisy nonlinear inverse problems such as Fourier phase retrieval and non-uniform deblurring. Code available at

Machine Learning,Artificial Intelligence,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper mainly discusses how to use diffusion models to solve the (non)linear inverse problems with noise. Existing diffusion models mainly focus on simple linear and noise-free inverse problems, while real-world problems are usually more complex. The paper proposes a new approximate method that effectively handles inverse problems with Gaussian and Poisson noise, as well as nonlinear problems. This method does not require strict measurement consistency projection steps but integrates diffusion sampling and manifold-constrained gradients to generate more ideal sample paths in a noisy environment.

The paper mentions that current methods fail to handle noisy measurements because the noise is usually amplified during the generation process, and these methods are ineffective when the measurement process is nonlinear. The authors apply the diffusion model to the frequency domain and utilize singular value decomposition (SVD) to handle noise, but the computational cost of SVD is high and not suitable for complex problems. Therefore, they propose a new method that can operate in the image domain, avoiding the computation of SVD, and applicable to various inverse problems such as image restoration, super-resolution, deblurring, and Fourier phase recovery.

The paper also demonstrates how the proposed method can be extended to deal with nonlinear inverse problems and adapt to different statistical characteristics of measurement noise. Through experiments, it is proven that this method has high quality and efficiency in solving inverse problems with noise.