Abstract:Diffusion models are emerging expressive generative models, in which a large number of time steps (inference steps) are required for a single image generation. To accelerate such tedious process, reducing steps uniformly is considered as an undisputed principle of diffusion models. We consider that such a uniform assumption is not the optimal solution in practice; i.e., we can find different optimal time steps for different models. Therefore, we propose to search the optimal time steps sequence and compressed model architecture in a unified framework to achieve effective image generation for diffusion models without any further training. Specifically, we first design a unified search space that consists of all possible time steps and various architectures. Then, a two stage evolutionary algorithm is introduced to find the optimal solution in the designed search space. To further accelerate the search process, we employ FID score between generated and real samples to estimate the performance of the sampled examples. As a result, the proposed method is (i).training-free, obtaining the optimal time steps and model architecture without any training process; (ii). orthogonal to most advanced diffusion samplers and can be integrated to gain better sample quality. (iii). generalized, where the searched time steps and architectures can be directly applied on different diffusion models with the same guidance scale. Experimental results show that our method achieves excellent performance by using only a few time steps, e.g. 17.86 FID score on ImageNet 64 $\times$ 64 with only four steps, compared to 138.66 with DDIM. The code is available at <a class="link-external link-https" href="https://github.com/lilijiangg/AutoDiffusion" rel="external noopener nofollow">this https URL</a>.

AdaDiff: Accelerating Diffusion Models through Step-Wise Adaptive Computation

AdaDiff: Adaptive Step Selection for Fast Diffusion.

AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Acceleration

Cambricon-D: Full-Network Differential Acceleration for Diffusion Models

AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising

Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference

PFDiff: Training-free Acceleration of Diffusion Models through the Gradient Guidance of Past and Future

FRDiff : Feature Reuse for Universal Training-free Acceleration of Diffusion Models

Improving Efficiency of Diffusion Models via Multi-Stage Framework and Tailored Multi-Decoder Architectures

Towards More Accurate Diffusion Model Acceleration with A Timestep Aligner

AdaDiffSR: Adaptive Region-aware Dynamic Acceleration Diffusion Model for Real-World Image Super-Resolution

DuoDiff: Accelerating Diffusion Models with a Dual-Backbone Approach

DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

Denoising Diffusion Step-aware Models

Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy

Accelerated Image-Aware Generative Diffusion Modeling

Accelerating Parallel Sampling of Diffusion Models

Diffusion Models Without Attention

Partially Conditioned Patch Parallelism for Accelerated Diffusion Model Inference

BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference

The Missing U for Efficient Diffusion Models