Abstract:Diffusion Probabilistic Models (DPMs) have demonstrated exceptional performance in generative tasks, but this comes at the expense of sampling efficiency. To enhance sampling speed without sacrificing quality, various distillation-based accelerated sampling algorithms have been recently proposed. However, they typically require significant additional training costs and model parameter storage, which limit their practical application. In this work, we propose PCA-based Adaptive Search (PAS), which optimizes existing solvers for DPMs with minimal learnable parameters and training costs. Specifically, we first employ PCA to obtain a few orthogonal unit basis vectors to span the high-dimensional sampling space, which enables us to learn just a set of coordinates to correct the sampling direction; furthermore, based on the observation that the cumulative truncation error exhibits an ``S''-shape, we design an adaptive search strategy that further enhances the sampling efficiency and reduces the number of stored parameters to approximately 10. Extensive experiments demonstrate that PAS can significantly enhance existing fast solvers in a plug-and-play manner with negligible costs. For instance, on CIFAR10, PAS requires only 12 parameters and less than 1 minute of training on a single NVIDIA A100 GPU to optimize the DDIM from 15.69 FID (NFE=10) to 4.37.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that although diffusion probability models (DPMs) perform well in generation tasks, their sampling efficiency is low. Specifically, the denoising process of DPMs usually requires hundreds or thousands of iteration steps, resulting in a very slow sampling speed, which seriously hinders their practical applications. In addition, the existing accelerated sampling algorithms either require a large amount of additional training costs and storage parameters, or accumulate excessive truncation errors during a small number of iterations, resulting in poor sampling effects. To solve these problems, the authors propose the PCA - based Adaptive Search (PAS) method, aiming to optimize the existing DPMs solvers with the minimum learnable parameters and training costs. The main contributions of PAS include: 1. **Proposing a new plug - and - play training paradigm**: Only about 10 parameters are required to significantly improve the performance of the existing fast DPM solvers, making the learning parameters and training costs almost negligible. 2. **Designing an adaptive search strategy**: Reducing the correction steps, further improving the sampling efficiency and reducing the number of storage parameters. 3. **Extensive experimental verification**: The effectiveness of the PAS method has been verified on multiple datasets, proving that it can significantly improve the image quality with almost no additional cost. The core idea of the PAS method is to use principal component analysis (PCA) to extract a few orthogonal unit basis vectors from high - dimensional sampling trajectories and learn the coordinates corresponding to these basis vectors to correct the sampling direction. In addition, observing that the cumulative truncation error shows an "S" - shaped trend, the authors design an adaptive search strategy to balance the sampling steps that need to be corrected and the truncation error, thereby further improving the sampling efficiency and reducing the number of required storage parameters. In conclusion, the PAS method provides an efficient and low - cost solution that can significantly accelerate the sampling speed of DPMs while maintaining high - quality generation results.

Diffusion Sampling Correction via Approximately 10 Parameters

Multi-Step Denoising Scheduled Sampling: Towards Alleviating Exposure Bias for Diffusion Models

Accelerating Diffusion Sampling with Optimized Time Steps

DC-Solver: Improving Predictor-Corrector Diffusion Sampler via Dynamic Compensation

A Unified Sampling Framework for Solver Searching of Diffusion Probabilistic Models

UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models

Fast Diffusion Probabilistic Model Sampling through the lens of Backward Error Analysis

DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics

DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps.

Accelerating Parallel Sampling of Diffusion Models

PFDiff: Training-free Acceleration of Diffusion Models through the Gradient Guidance of Past and Future

DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models

Optimizing DDPM Sampling with Shortcut Fine-Tuning

SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models

On Accelerating Diffusion-Based Sampling Process via Improved Integration Approximation

Faster Sampling without Isoperimetry via Diffusion-based Monte Carlo

Accelerating Convergence of Score-Based Diffusion Models, Provably

Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity

Boosting Diffusion Models with an Adaptive Momentum Sampler

Simple and Fast Distillation of Diffusion Models

DPM-OT: A New Diffusion Probabilistic Model Based on Optimal Transport