Abstract:Score-based diffusion models have significantly advanced high-dimensional data generation across various domains, by learning a denoising oracle (or score) from datasets. From a Bayesian perspective, they offer a realistic modeling of data priors and facilitate solving inverse problems through posterior sampling. Although many heuristic methods have been developed recently for this purpose, they lack the quantitative guarantees needed in many scientific applications. In this work, we introduce the \textit{tilted transport} technique, which leverages the quadratic structure of the log-likelihood in linear inverse problems in combination with the prior denoising oracle to transform the original posterior sampling problem into a new `boosted' posterior that is provably easier to sample from. We quantify the conditions under which this boosted posterior is strongly log-concave, highlighting the dependencies on the condition number of the measurement matrix and the signal-to-noise ratio. The resulting posterior sampling scheme is shown to reach the computational threshold predicted for sampling Ising models [Kunisky'23] with a direct analysis, and is further validated on high-dimensional Gaussian mixture models and scalar field $\varphi^4$ models.

What problem does this paper attempt to address?

The paper primarily aims to address the issue of posterior sampling based on diffusion models in the field of high-dimensional data generation, particularly how to effectively sample from the posterior distribution in the context of linear inverse problems. The paper first introduces the advancements of score-based diffusion models in high-dimensional data generation and points out that these models, by learning denoising operators (or scores), can provide realistic modeling of data priors and help solve posterior sampling in inverse problems. However, existing heuristic methods for posterior sampling lack the quantitative guarantees required for many scientific applications. To address this issue, the authors introduce a technique called "Tilted Transport." This technique leverages the quadratic structure of the likelihood function in linear inverse problems combined with the prior denoising operator, transforming the original posterior sampling problem into a new, theoretically easier-to-sample "enhanced" posterior distribution. Specifically, when the condition number of the measurement matrix and the signal-to-noise ratio meet certain conditions, the enhanced posterior distribution can be proven to be strongly log-concave, making efficient sampling via Langevin dynamics possible. Furthermore, the paper quantifies the conditions under which the enhanced posterior distribution becomes strongly log-concave and demonstrates the dependency of these conditions on the condition number of the measurement matrix and the signal-to-noise ratio. The experimental section validates the effectiveness of the proposed Tilted Transport method in high-dimensional Gaussian mixture models and the scalar field φ^4 model, including achieving the predicted computational threshold of the Ising model. In summary, the goal of this paper is to improve the efficiency and accuracy of posterior sampling based on diffusion models by proposing a new Tilted Transport technique, especially when dealing with high-dimensional linear inverse problems. This method not only provides theoretical guarantees but also shows superiority in practical case studies.

Posterior Sampling with Denoising Oracles via Tilted Transport

Diffusion Posterior Sampling for General Noisy Inverse Problems

Posterior Sampling in High Dimension via Diffusion Processes

Diffusion Model Based Posterior Sampling for Noisy Linear Inverse Problems

Tweedie Moment Projected Diffusions For Inverse Problems

A New Perspective On Denoising Based On Optimal Transport

Taming Score-Based Diffusion Priors for Infinite-Dimensional Nonlinear Inverse Problems

Divide-and-Conquer Posterior Sampling for Denoising Diffusion Priors

Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance

Enhancing Diffusion Posterior Sampling for Inverse Problems by Integrating Crafted Measurements

Stochastic Localization via Iterative Posterior Sampling

Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstruction

Variational Diffusion Posterior Sampling with Midpoint Guidance

Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing

Posterior sampling via Langevin dynamics based on generative priors

Principled Probabilistic Imaging using Diffusion Models as Plug-and-Play Priors

Plug-and-Play Posterior Sampling under Mismatched Measurement and Prior Models

Denoising Diffusion Samplers

Amortized Posterior Sampling with Diffusion Prior Distillation

Listening to the Noise: Blind Denoising with Gibbs Diffusion

Removing Structured Noise with Diffusion Models