Abstract:Accelerating the sampling speed of diffusion models remains a significant challenge. Recent score distillation methods distill a heavy teacher model into a student generator to achieve one-step generation, which is optimized by calculating the difference between the two score functions on the samples generated by the student model. However, there is a score mismatch issue in the early stage of the distillation process, because existing methods mainly focus on using the endpoint of pre-trained diffusion models as teacher models, overlooking the importance of the convergence trajectory between the student generator and the teacher model. To address this issue, we extend the score distillation process by introducing the entire convergence trajectory of teacher models and propose Distribution Backtracking Distillation (DisBack). DisBask is composed of two stages: Degradation Recording and Distribution Backtracking. Degradation Recording is designed to obtain the convergence trajectory of the teacher model, which records the degradation path from the trained teacher model to the untrained initial student generator. The degradation path implicitly represents the teacher model's intermediate distributions, and its reverse can be viewed as the convergence trajectory from the student generator to the teacher model. Then Distribution Backtracking trains a student generator to backtrack the intermediate distributions along the path to approximate the convergence trajectory of teacher models. Extensive experiments show that DisBack achieves faster and better convergence than the existing distillation method and accomplishes comparable generation performance, with FID score of 1.38 on ImageNet 64x64 dataset. Notably, DisBack is easy to implement and can be generalized to existing distillation methods to boost performance. Our code is publicly available on <a class="link-external link-https" href="https://github.com/SYZhang0805/DisBack" rel="external noopener nofollow">this https URL</a>.

Target-Driven Distillation: Consistency Distillation with Target Timestep Selection and Decoupled Guidance

Improved Distribution Matching Distillation for Fast Image Synthesis

Distribution Backtracking Builds A Faster Convergence Trajectory for Diffusion Distillation

One-step Diffusion with Distribution Matching Distillation

Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mapping

Relational Diffusion Distillation for Efficient Image Generation

Tree-like Decision Distillation – Supplementary Material –

Not All Steps Are Created Equal: Selective Diffusion Distillation for Image Manipulation

Catch-Up Distillation: You Only Need to Train Once for Accelerating Sampling

Plug-and-Play Diffusion Distillation

Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps

Simple and Fast Distillation of Diffusion Models

Multi-student Diffusion Distillation for Better One-step Generators

Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis

SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation

Distilling Diffusion Models into Conditional GANs

SFDDM: Single-fold Distillation for Diffusion models

TLCM: Training-efficient Latent Consistency Model for Image Generation with 2-8 Steps

EM Distillation for One-step Diffusion Models

ACT-Diffusion: Efficient Adversarial Consistency Training for One-step Diffusion Models

DFGPD: a new distillation framework with global and positional distillation