Abstract:The rapid deployment of generative language models (LMs) has raised concerns about social biases affecting the well-being of diverse consumers. The extant literature on generative LMs has primarily examined bias via explicit identity prompting. However, prior research on bias in earlier language-based technology platforms, including search engines, has shown that discrimination can occur even when identity terms are not specified explicitly. Studies of bias in LM responses to open-ended prompts (where identity classifications are left unspecified) are lacking and have not yet been grounded in end-consumer harms. Here, we advance studies of generative LM bias by considering a broader set of natural use cases via open-ended prompting. In this "laissez-faire" setting, we find that synthetically generated texts from five of the most pervasive LMs (ChatGPT3.5, ChatGPT4, Claude2.0, Llama2, and PaLM2) perpetuate harms of omission, subordination, and stereotyping for minoritized individuals with intersectional race, gender, and/or sexual orientation identities (AI/AN, Asian, Black, Latine, MENA, NH/PI, Female, Non-binary, Queer). We find widespread evidence of bias to an extent that such individuals are hundreds to thousands of times more likely to encounter LM-generated outputs that portray their identities in a subordinated manner compared to representative or empowering portrayals. We also document a prevalence of stereotypes (e.g. perpetual foreigner) in LM-generated outputs that are known to trigger psychological harms that disproportionately affect minoritized individuals. These include stereotype threat, which leads to impaired cognitive performance and increased negative self-perception. Our findings highlight the urgent need to protect consumers from discriminatory harms caused by language models and invest in critical AI education programs tailored towards empowering diverse consumers.

Defining and Evaluating Fair Natural Language Generation

Evaluating Gender Bias in Natural Language Inference

Generalizing Fairness to Generative Language Models via Reformulation of Non-discrimination Criteria

Addressing Both Statistical and Causal Gender Fairness in NLP Models

Fairness Definitions in Language Models Explained

Mitigating Gender Bias in Natural Language Processing: Literature Review

Fairness Certification for Natural Language Processing and Large Language Models

A Survey on Gender Bias in Natural Language Processing

Bridging Fairness and Environmental Sustainability in Natural Language Processing

FairPair: A Robust Evaluation of Biases in Language Models through Paired Perturbations

Gender Bias in Neural Natural Language Processing

On the Interplay between Fairness and Explainability

Gender Bias in Large Language Models across Multiple Languages

Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias

"One-Size-Fits-All"? Examining Expectations around What Constitute "Fair" or "Good" NLG System Behaviors

Using Natural Sentences for Understanding Biases in Language Models

Social Biases in Automatic Evaluation Metrics for NLG

Bias and Fairness in Large Language Models: A Survey

Laissez-Faire Harms: Algorithmic Biases in Generative Language Models

''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English Text

Fairness in Language Models Beyond English: Gaps and Challenges