Stable Diffusion Exposed: Gender Bias from Prompt to Image

Yankun Wu,Yuta Nakashima,Noa Garcia

2024-08-11

Abstract:Several studies have raised awareness about social biases in image generative models, demonstrating their predisposition towards stereotypes and imbalances. This paper contributes to this growing body of research by introducing an evaluation protocol that analyzes the impact of gender indicators at every step of the generation process on Stable Diffusion images. Leveraging insights from prior work, we explore how gender indicators not only affect gender presentation but also the representation of objects and layouts within the generated images. Our findings include the existence of differences in the depiction of objects, such as instruments tailored for specific genders, and shifts in overall layouts. We also reveal that neutral prompts tend to produce images more aligned with masculine prompts than their feminine counterparts. We further explore where bias originates through representational disparities and how it manifests in the images via prompt-image dependencies, and provide recommendations for developers and users to mitigate potential bias in image generation.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the gender bias in the Stable Diffusion image - generation model. Specifically, the authors introduced an evaluation protocol to analyze the influence of gender - indicating words on Stable Diffusion images during the generation process. They explored that gender - indicating words not only affect gender representation but also the representation of objects and layout in the generated images.

### Main Research Questions

1. **RQ1**: Are the images generated by neutral prompts more similar to those generated by male prompts than to those generated by female prompts? If so, why?
2. **RQ2**: Are there significant differences in the frequency of object occurrence in the images according to the gender specified in the prompts? If there are differences, are the objects generated by neutral prompts closer to those generated by male or female prompts?
3. **RQ3**: Does the gender in the input prompt affect the prompt - image dependency relationship in Stable Diffusion? If so, which prompt - image dependency relationships are more likely to be affected?

### Research Methods

To answer the above questions, the authors used the following methods:

1. **Triplet Prompt Generation**: Generate neutral prompts from natural - language sentences and generate corresponding female and male prompts by replacing gender - indicating words (such as replacing "person" with "woman" or "man").
2. **Image Generation**: Use the Stable Diffusion model to convert prompts into images, tracking the entire generation process from the prompt space to the denoising space and then to the image space.
3. **Representation Difference Analysis**: Compare the differences between images generated by different - gender prompts by calculating metrics such as cosine similarity.
4. **Object Co - occurrence Analysis**: Use a visual foundation model to detect objects in the generated images and analyze the co - occurrence of these objects under different - gender prompts.
5. **Bias Scoring**: Calculate the bias score of each object to quantify its degree of bias towards male or female prompts.

### Key Findings

1. **Neutral Prompts Are Biased Towards Males**: Whether in the prompt space, the denoising space, or the image space, the images generated by neutral prompts are closer to those generated by male prompts.
2. **Objects Not Explicitly Mentioned Have Gender Tendencies**: For objects not explicitly mentioned in the prompts, the generation rates are different under different - gender prompts. For example, some clothes (such as ties, aprons) are obviously biased towards a certain gender.
3. **Gender Bias Spreads to the Entire Image**: Gender bias is not limited to the representation of people but also permeates the entire image, affecting the generated objects and their layout.

### Conclusions and Recommendations

The authors suggest that developers and users take measures to mitigate the impact of this gender bias, such as improving the diversity and balance of the data set and optimizing the training process of the model. Through these efforts, gender stereotypes and other unfair phenomena in the generated images can be reduced.

### Formula Summary

- Cosine Similarity Formula:
\[
s(P, P')=\frac{1}{|P|} \sum_{p_{i}, p'_{i}} \cos(t, t')
\]
where \(|P|\) is the number of elements in a given set, \(\cos(·, ·)\) represents cosine similarity, and \(t\) and \(t'\) correspond to the text embeddings of prompts \(p_{i}\) and \(p'_{i}\) respectively.

- Object Co - occurrence Similarity Formula:
\[
s_{O}(P, P')=\frac{1}{|P|} \sum_{p_{i}, p'_{i}} \cos(v_{i}, v'_{i})
\]
where \(v_{i}\) and \(v'_{i}\) are from the object co - occurrence vectors of prompts \(p_{i}\) and \(p'_{i}\) respectively.

- Bias Scoring Formula:
\[
BS(o)=\frac{C(o, P_{m})}{C(o, P_{m})+\frac{|P_{m}|}{|P_{f}|} C(o, P_{f})}
\]
where \(C(o, P_{m})\) and \(C(o, P_{f})\) represent the co - occurrence times of object \(o\) in the male - prompt set \(P_{m}\) and the female - prompt set \(P_{f}\) respectively.