Abstract:As one of the most successful generative models, diffusion models have demonstrated remarkable efficacy in synthesizing high-quality images. These models learn the underlying high-dimensional data distribution in an unsupervised manner. Despite their success, diffusion models are highly data-driven and prone to inheriting the imbalances and biases present in real-world data. Some studies have attempted to address these issues by designing text prompts for known biases or using bias labels to construct unbiased data. While these methods have shown improved results, real-world scenarios often contain various unknown biases, and obtaining bias labels is particularly challenging. In this paper, we emphasize the necessity of mitigating bias in pre-trained diffusion models without relying on auxiliary bias annotations. To tackle this problem, we propose a framework, InvDiff, which aims to learn invariant semantic information for diffusion guidance. Specifically, we propose identifying underlying biases in the training data and designing a novel debiasing training objective. Then, we employ a lightweight trainable module that automatically preserves invariant semantic information and uses it to guide the diffusion model's sampling process toward unbiased outcomes simultaneously. Notably, we only need to learn a small number of parameters in the lightweight learnable module without altering the pre-trained diffusion model. Furthermore, we provide a theoretical guarantee that the implementation of InvDiff is equivalent to reducing the error upper bound of generalization. Extensive experimental results on three publicly available benchmarks demonstrate that InvDiff effectively reduces biases while maintaining the quality of image generation. Our code is available at <a class="link-external link-https" href="https://github.com/Hundredl/InvDiff" rel="external noopener nofollow">this https URL</a>.

Estimation of image bias field by diffusion

Behavioral Analysis of Anisotropic Diffusion in Image Processing .

Image Segmentation Based on the Hybrid Bias Field Correction

Analysis and Design of Anisotropic Diffusion for Image Processing

A Belief Propagation Algorithm For Bias Field Estimation And Image Segmentation

A Robust Parametric Method for Bias Field Estimation and Segmentation of MR Images

A Multiphase Chan-Vese Model Using Edge Information and Bias Field Correction for Image Segmentation

A New Active Contour Model Driven by Pre-Fitting Bias Field Estimation and Clustering Technique for Image Segmentation.

An MR Image Segmentation Algorithm Based on Bias Field Correction

Active contour model based on adaptive segmentation and bias field estimation

Thermal Radiation Bias Correction for Infrared Images Using Huber Function Based Loss

InvDiff: Invariant Guidance for Bias Mitigation in Diffusion Models

A fuzzy C-means based algorithm for bias field estimation and segmentation of MR images

A level set method based on additive bias correction for image segmentation

Enhanced Gradient-Domain LMS Method for Bias Field Correction in Infrared Images

Bias Correction for Magnetic Resonance Images Via Joint Entropy Regularization

Biases of Estimated Signals in X-Ray Analyzer-Based Imaging

A variational level set approach to segmentation and bias correction of images with intensity inhomogeneity.

Unmasking Bias in Diffusion Model Training

A New Approach for Correcting Bias Field in MR Images Using Singularity Spectrum Analysis

A RVR-based Method for Bias Field Estimation in Brain Magnetic Resonance Images Segmentation