Abstract:Diffusion models have shown remarkable performance on many generative tasks. Despite recent success, most diffusion models are restricted in that they only allow linear transformation of the data distribution. In contrast, broader family of transformations can potentially help train generative distributions more efficiently, simplifying the reverse process and closing the gap between the true negative log-likelihood and the variational approximation. In this paper, we present Neural Diffusion Models (NDMs), a generalization of conventional diffusion models that enables defining and learning time-dependent non-linear transformations of data. We show how to optimise NDMs using a variational bound in a simulation-free setting. Moreover, we derive a time-continuous formulation of NDMs, which allows fast and reliable inference using off-the-shelf numerical ODE and SDE solvers. Finally, we demonstrate the utility of NDMs with learnable transformations through experiments on standard image generation benchmarks, including CIFAR-10, downsampled versions of ImageNet and CelebA-HQ. NDMs outperform conventional diffusion models in terms of likelihood and produce high-quality samples.

What problem does this paper attempt to address?

The paper mainly focuses on addressing the following issues: 1. **Enhancing the Flexibility and Performance of Diffusion Models**: Existing diffusion models perform well in generative tasks, but most models only allow linear transformations of data distributions, which limits their ability to adapt to specific tasks or data. The paper introduces a new framework—Neural Diffusion Models (NDMs)—to overcome this limitation, enabling the models to define and learn nonlinear, time-dependent data transformations. 2. **Improving Likelihood Estimation**: The primary motivation behind NDMs is to improve likelihood and density estimation, which is crucial for applications such as compression, semi-supervised learning, and adversarial purification. 3. **Simplifying the Reverse Process**: By introducing flexible and learnable data transformations, NDMs simplify the task of the reverse (generative) process, thereby enhancing the quality of the model. 4. **Achieving Efficient Training and Inference**: The paper proposes a variational objective function to optimize NDMs, and this objective function is an upper bound estimate of the negative log-likelihood, making the training process possible without simulating all latent variables, thus achieving a "simulation-free" training method. Furthermore, the corresponding objective function under continuous-time models, as well as the associated stochastic differential equations (SDE) and ordinary differential equations (ODE), are derived, allowing for fast and reliable inference using off-the-shelf numerical solvers. 5. **Expanding the Capabilities of Existing Diffusion Models**: The paper demonstrates how NDMs generalize various existing diffusion models and validates their superior performance on multiple image generation benchmarks through experiments, including the MNIST, CIFAR-10, downsampled ImageNet, and CelebA-HQ datasets. In summary, the main contribution of this paper is the proposal of a more universal and flexible diffusion model framework, which not only improves the generative quality of the model but also holds significant theoretical and practical value.

Neural Diffusion Models

Where to Diffuse, How to Diffuse, and How to Get Back: Automated Learning for Multivariate Diffusions

Image Neural Field Diffusion Models

Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modelling

Neural Network Parameter Diffusion

Neural Residual Diffusion Models for Deep Scalable Vision Generation

The Uncanny Valley: A Comprehensive Analysis of Diffusion Models

Non-Uniform Diffusion Models

An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization

An overview of diffusion models for generative artificial intelligence

Diffusion Models For Multi-Modal Generative Modeling

Diffusion models as probabilistic neural operators for recovering unobserved states of dynamical systems

Non-Normal Diffusion Models

Diffusion Model Conditioning on Gaussian Mixture Model and Negative Gaussian Mixture Gradient

GUD: Generation with Unified Diffusion

Nested Diffusion Processes for Anytime Image Generation

Maximum Likelihood Training of Implicit Nonlinear Diffusion Models

Stable Diffusion with Continuous-time Neural Network

HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion

Observation-Guided Diffusion Probabilistic Models

Diffusion Models in Vision: A Survey