Abstract:Score-based generative models (SGMs) have recently emerged as a promising class of generative models. However, a fundamental limitation is that their sampling process is slow due to a need for many (\eg, $2000$) iterations of sequential computations. An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation. We assault this problem to the ill-conditioned issues of the Langevin dynamics and reverse diffusion in the sampling process. Under this insight, we propose a model-agnostic {\bf\em preconditioned diffusion sampling} (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem. PDS alters the sampling process of a vanilla SGM at marginal extra computation cost, and without model retraining. Theoretically, we prove that PDS preserves the output distribution of the SGM, no risk of inducing systematical bias to the original sampling process. We further theoretically reveal a relation between the parameter of PDS and the sampling iterations,easing the parameter estimation under varying sampling iterations. Extensive experiments on various image datasets with a variety of resolutions and diversity validate that our PDS consistently accelerates off-the-shelf SGMs whilst maintaining the synthesis quality. In particular, PDS can accelerate by up to $29\times$ on more challenging high resolution (1024$\times$1024) image generation. Compared with the latest generative models (\eg, CLD-SGM, DDIM, and Analytic-DDIM), PDS can achieve the best sampling quality on CIFAR-10 at a FID score of 1.99. Our code is made publicly available to foster any further research <a class="link-external link-https" href="https://github.com/fudan-zvg/PDS" rel="external noopener nofollow">this https URL</a>.

Approximated Anomalous Diffusion: Gaussian Mixture Score-based Generative Models

Preconditioned Score-based Generative Models

Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

Exploring Guided Sampling of Conditional GANs

Convergence of score-based generative modeling for general data distributions

A Complete Recipe for Diffusion Generative Models

Accelerating Score-based Generative Models with Preconditioned Diffusion Sampling

Closed-Form Diffusion Models

Accelerating Score-based Generative Models for High-Resolution Image Synthesis

Gaussian Mixture Solvers for Diffusion Models

Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions

Score-Based Generative Modeling with Critically-Damped Langevin Diffusion

Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models

Score-based Generative Models with Adaptive Momentum

A Score-Based Density Formula, with Applications in Diffusion Generative Models

Posterior sampling via Langevin dynamics based on generative priors

Evaluating the design space of diffusion-based generative models

Generative Modelling with High-Order Langevin Dynamics

Glauber Generative Model: Discrete Diffusion Models via Binary Classification

Diffusion-Model-Assisted Supervised Learning of Generative Models for Density Estimation