From Melting Pots to Misrepresentations: Exploring Harms in Generative AI

Sanjana Gautam,Pranav Narayanan Venkit,Sourojit Ghosh

2024-03-16

Abstract:With the widespread adoption of advanced generative models such as Gemini and GPT, there has been a notable increase in the incorporation of such models into sociotechnical systems, categorized under AI-as-a-Service (AIaaS). Despite their versatility across diverse sectors, concerns persist regarding discriminatory tendencies within these models, particularly favoring selected `majority' demographics across various sociodemographic dimensions. Despite widespread calls for diversification of media representations, marginalized racial and ethnic groups continue to face persistent distortion, stereotyping, and neglect within the AIaaS context. In this work, we provide a critical summary of the state of research in the context of social harms to lead the conversation to focus on their implications. We also present open-ended research questions, guided by our discussion, to help define future research pathways.

Human-Computer Interaction,Artificial Intelligence,Computers and Society,Machine Learning

What problem does this paper attempt to address?

The paper primarily explores the issues of bias and harm present in Generative AI (GAI) services. The authors focus on how these systems exhibit unjust performance across different social groups, particularly among minority and marginalized groups. Specifically, the paper aims to address the following key issues:

1. **Current State of Bias and Harm**: The paper analyzes the types of biases present in current generative AI models, particularly how these models tend to reinforce social stereotypes and exhibit uneven performance when dealing with different social groups.

2. **Specific Case Studies**: Through examples, such as the bias shown when using Google's Imagen 2 image generation model to generate images of "upper-class families," and the limitations or erroneous responses encountered when attempting to generate images of families from other socioeconomic classes. This reveals the differences in service quality among different socioeconomic groups and the marginalization of certain groups.

3. **Ethical Design and Future Directions**: The paper proposes a community-centered design approach to improve generative AI models, emphasizing the importance of considering ethical impacts during the development process. Additionally, the paper raises several open research questions aimed at guiding future research directions, including how to investigate specific biases in models, identify new forms of harm, and hold developers accountable for the harm caused.

In summary, the core objective of this paper is to reveal the social biases and potential harms present in generative AI services and to advocate for more responsible methods in designing and evaluating such technologies to promote fairness and inclusivity.