Abstract:Recent approaches have shown promises distilling diffusion models into efficient one-step generators. Among them, Distribution Matching Distillation (DMD) produces one-step generators that match their teacher in distribution, without enforcing a one-to-one correspondence with the sampling trajectories of their teachers. However, to ensure stable training, DMD requires an additional regression loss computed using a large set of noise-image pairs generated by the teacher with many steps of a deterministic sampler. This is costly for large-scale text-to-image synthesis and limits the student's quality, tying it too closely to the teacher's original sampling paths. We introduce DMD2, a set of techniques that lift this limitation and improve DMD training. First, we eliminate the regression loss and the need for expensive dataset construction. We show that the resulting instability is due to the fake critic not estimating the distribution of generated samples accurately and propose a two time-scale update rule as a remedy. Second, we integrate a GAN loss into the distillation procedure, discriminating between generated samples and real images. This lets us train the student model on real data, mitigating the imperfect real score estimation from the teacher model, and enhancing quality. Lastly, we modify the training procedure to enable multi-step sampling. We identify and address the training-inference input mismatch problem in this setting, by simulating inference-time generator samples during training time. Taken together, our improvements set new benchmarks in one-step image generation, with FID scores of 1.28 on ImageNet-64x64 and 8.35 on zero-shot COCO 2014, surpassing the original teacher despite a 500X reduction in inference cost. Further, we show our approach can generate megapixel images by distilling SDXL, demonstrating exceptional visual quality among few-step methods.

SDXL-Lightning: Progressive Adversarial Diffusion Distillation

Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation

AnimateDiff-Lightning: Cross-Model Diffusion Distillation

Progressive Knowledge Distillation Of Stable Diffusion XL Using Layer Level Loss

Data-Free Adversarial Distillation

SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis

Not All Steps Are Created Equal: Selective Diffusion Distillation for Image Manipulation

Distilling Diffusion Models into Conditional GANs

One-step Diffusion with Distribution Matching Distillation

Multi-student Diffusion Distillation for Better One-step Generators

Efficient Dataset Distillation via Diffusion-Driven Patch Selection for Improved Generalization

Latent Dataset Distillation with Diffusion Models

Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis

Relational Diffusion Distillation for Efficient Image Generation

AddSR: Accelerating Diffusion-based Blind Super-Resolution with Adversarial Diffusion Distillation

CAD: Photorealistic 3D Generation via Adversarial Distillation

Learning Differentially Private Diffusion Models via Stochastic Adversarial Distillation

LAPTOP-Diff: Layer Pruning and Normalized Distillation for Compressing Diffusion Models

Improved Distribution Matching Distillation for Fast Image Synthesis

One Step Diffusion-based Super-Resolution with Time-Aware Distillation

UniFL: Improve Latent Diffusion Model via Unified Feedback Learning