Abstract:In the field of healthcare, the acquisition of sample is usually restricted by multiple considerations, including cost, labor- intensive annotation, privacy concerns, and radiation hazards, therefore, synthesizing images-of-interest is an important tool to data augmentation. Diffusion models have recently attained state-of-the-art results in various synthesis tasks, and embedding energy functions has been proved that can effectively guide the pre-trained model to synthesize target samples. However, we notice that current method development and validation are still limited to improving indicators, such as Fréchet Inception Distance score (FID) and Inception Score (IS), and have not provided deeper investigations on downstream tasks, like disease grading and diagnosis. Moreover, existing classifier guidance which can be regarded as a special case of energy function can only has a singular effect on altering the distribution of the synthetic dataset. This may contribute to in-distribution synthetic sample that has limited help to downstream model optimization. All these limitations remind that we still have a long way to go to achieve controllable generation. In this work, we first conducted an analysis on previous guidance as well as its contributions on further applications from the perspective of data distribution. To synthesize samples which can help downstream applications, we then introduce uncertainty guidance in each sampling step and design an uncertainty-guided diffusion models. Extensive experiments on four medical datasets, with ten classic networks trained on the augmented sample sets provided a comprehensive evaluation on the practical contributions of our methodology. Furthermore, we provide a theoretical guarantee for general gradient guidance in diffusion models, which would benefit future research on investigating other forms of measurement guidance for specific generative tasks. Codes and models are available at: https://github.com/yangqy1110/MGDM.

Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention

Self-Guidance: Boosting Flow and Diffusion Generation on Their Own

Improving Sample Quality of Diffusion Models Using Self-Attention Guidance

Segmentation-Free Guidance for Text-to-Image Diffusion Models

Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models

Energy-Based Cross Attention for Bayesian Context Update in Text-to-Image Diffusion Models

The Unreasonable Effectiveness of Guidance for Diffusion Models

Semantic Guidance Tuning for Text-To-Image Diffusion Models

EP-CFG: Energy-Preserving Classifier-Free Guidance

Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidance

Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning

Spatiotemporal Skip Guidance for Enhanced Video Diffusion Sampling

Elucidating The Design Space of Classifier-Guided Diffusion Generation

Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory

Self-Guided Diffusion Models

End-to-End Diffusion Latent Optimization Improves Classifier Guidance

Universal Guidance for Diffusion Models

Measurement Guidance in Diffusion Models: Insight from Medical Image Synthesis

Towards Accurate Guided Diffusion Sampling through Symplectic Adjoint Method

Diffusion Self-Guidance for Controllable Image Generation

Layout Control and Semantic Guidance with Attention Loss Backward for T2I Diffusion Model