Abstract:Denoising diffusion probabilistic models (DDPM) have shown impressive performance in various domains as a class of deep generative models. In this paper, we introduce the Mixture noise-based DDPM (Mix-DDPM), which considers the Markov diffusion posterior as a Gaussian mixture model. Specifically, Mix-DDPM randomly selects a Gaussian component and then adds the chosen Gaussian noise, which can be demonstrated as a more efficient way to perturb the signals into a simple known distribution. We further define the reverse probabilistic model as a parameterized Gaussian mixture kernel. Due to the intractability in calculating the KL divergence between Gaussian mixture models, we derive a variational bound to maximize the likelihood, offering a concise formulation for optimizing the denoising model and valuable insights for designing the sampling strategies. Our theoretical derivation highlights that Mix-DDPM need only shift image which requires the inclusion of a global stochastic offset in both the diffusion and reverse processes , which can be efficiently implemented with just several lines of code. The global stochastic offset effectively fits a Gaussian mixture distribution enhancing the degrees of freedom of the entire diffusion model. Furthermore, we present three streamlined sampling strategies that interface with diverse fast dedicated solvers for diffusion ordinary differential equations, boosting the efficacy of image representation in the sampling phase and alleviating the issue of slow generation speed, thereby enhancing both efficiency and accuracy. Extensive experiments on benchmark datasets demonstrate the effectiveness of Mix-DDPM and its superiority over the original DDPM.

Optimizing Noise Schedules of Generative Models in High Dimensionss

Edge-preserving noise for diffusion models

Error Bound Based Noise Schedule Design in Diffusion Models

One Noise to Rule Them All: Learning a Unified Model of Spatially-Varying Noise Patterns

An analysis of the noise schedule for score-based generative models

Evaluating the design space of diffusion-based generative models

On the Noise Scheduling for Generating Plausible Designs with Diffusion Models

Not All Noises Are Created Equally:Diffusion Noise Selection and Optimization

Using Ornstein-Uhlenbeck Process to understand Denoising Diffusion Probabilistic Model and its Noise Schedules

Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models

Heavy-tailed denoising score matching

Discrete vs. Continuous Trade-offs for Generative Models

Score-Optimal Diffusion Schedules

Exploring the Optimal Choice for Generative Processes in Diffusion Models: Ordinary vs Stochastic Differential Equations

Align Your Steps: Optimizing Sampling Schedules in Diffusion Models

High Noise Scheduling is a Must

On the Theoretical Properties of Noise Correlation in Stochastic Optimization

Mix-DDPM: Enhancing Diffusion Models Through Fitting Mixture Noise with Global Stochastic Offset