Chatbot AI makes racist judgements on the basis of dialect

Elizabeth Gibney

DOI: https://doi.org/10.1038/d41586-024-00779-1

IF: 64.8

2024-03-14

Nature

Abstract:Some large language models harbour hidden biases that cannot be removed using standard methods. Some large language models harbour hidden biases that cannot be removed using standard methods.

multidisciplinary sciences

What problem does this paper attempt to address?

AI & robotics briefing: LLMs harbour hidden racism

Katrina Krämer

DOI: https://doi.org/10.1038/d41586-024-00856-5

IF: 64.8

2024-03-21

Nature

Abstract:Chatbots make racist judgements on the basis of dialect. Plus, cyborg jellyfish could monitor the changing seas and how AI designed a drug. Chatbots make racist judgements on the basis of dialect. Plus, cyborg jellyfish could monitor the changing seas and how AI designed a drug.

multidisciplinary sciences
AI generates covertly racist decisions about people based on their dialect

Valentin Hofmann,Pratyusha Ria Kalluri,Dan Jurafsky,Sharese King

DOI: https://doi.org/10.1038/s41586-024-07856-5

IF: 64.8

2024-08-31

Nature

Abstract:Hundreds of millions of people now interact with language models, with uses ranging from help with writing 1,2 to informing hiring decisions 3 . However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans 4,5,6,7 . Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement 8,9 . It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models' overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.

multidisciplinary sciences
Dialect prejudice predicts AI decisions about people's character, employability, and criminality

Valentin Hofmann,Pratyusha Ria Kalluri,Dan Jurafsky,Sharese King

2024-03-02

Abstract:Hundreds of millions of people now interact with language models, with uses ranging from serving as a writing aid to informing hiring decisions. Yet these language models are known to perpetuate systematic racial prejudices, making their judgments biased in problematic ways about groups like African Americans. While prior research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice: we extend research showing that Americans hold raciolinguistic stereotypes about speakers of African American English and find that language models have the same prejudice, exhibiting covert stereotypes that are more negative than any human stereotypes about African Americans ever experimentally recorded, although closest to the ones from before the civil rights movement. By contrast, the language models' overt stereotypes about African Americans are much more positive. We demonstrate that dialect prejudice has the potential for harmful consequences by asking language models to make hypothetical decisions about people, based only on how they speak. Language models are more likely to suggest that speakers of African American English be assigned less prestigious jobs, be convicted of crimes, and be sentenced to death. Finally, we show that existing methods for alleviating racial bias in language models such as human feedback training do not mitigate the dialect prejudice, but can exacerbate the discrepancy between covert and overt stereotypes, by teaching language models to superficially conceal the racism that they maintain on a deeper level. Our findings have far-reaching implications for the fair and safe employment of language technology.

Computation and Language,Artificial Intelligence,Computers and Society
Computer says 'no': Exploring systemic bias in ChatGPT using an audit approach

Louis Lippens

DOI: https://doi.org/10.1016/j.chbah.2024.100054

2024-02-12

Abstract:Large language models offer significant potential for increasing labour productivity, such as streamlining personnel selection, but raise concerns about perpetuating systemic biases embedded into their pre-training data. This study explores the potential ethnic and gender bias of ChatGPT, a chatbot producing human-like responses to language tasks, in assessing job applicants. Using the correspondence audit approach from the social sciences, I simulated a CV screening task with 34,560 vacancy-CV combinations where the chatbot had to rate fictitious applicant profiles. Comparing ChatGPT's ratings of Arab, Asian, Black American, Central African, Dutch, Eastern European, Hispanic, Turkish, and White American male and female applicants, I show that ethnic and gender identity influence the chatbot's evaluations. Ethnic discrimination is more pronounced than gender discrimination and mainly occurs in jobs with favourable labour conditions or requiring greater language proficiency. In contrast, gender discrimination emerges in gender-atypical roles. These findings suggest that ChatGPT's discriminatory output reflects a statistical mechanism echoing societal stereotypes. Policymakers and developers should address systemic bias in language model-driven applications to ensure equitable treatment across demographic groups. Practitioners should practice caution, given the adverse impact these tools can (re)produce, especially in selection decisions involving humans.

General Economics
Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination

Eve Fleisig,Genevieve Smith,Madeline Bossi,Ishita Rustagi,Xavier Yin,Dan Klein

2024-09-17

Abstract:We present a large-scale study of linguistic bias exhibited by ChatGPT covering ten dialects of English (Standard American English, Standard British English, and eight widely spoken non-"standard" varieties from around the world). We prompted GPT-3.5 Turbo and GPT-4 with text by native speakers of each variety and analyzed the responses via detailed linguistic feature annotation and native speaker evaluation. We find that the models default to "standard" varieties of English; based on evaluation by native speakers, we also find that model responses to non-"standard" varieties consistently exhibit a range of issues: stereotyping (19% worse than for "standard" varieties), demeaning content (25% worse), lack of comprehension (9% worse), and condescending responses (15% worse). We also find that if these models are asked to imitate the writing style of prompts in non-"standard" varieties, they produce text that exhibits lower comprehension of the input and is especially prone to stereotyping. GPT-4 improves on GPT-3.5 in terms of comprehension, warmth, and friendliness, but also exhibits a marked increase in stereotyping (+18%). The results indicate that GPT-3.5 Turbo and GPT-4 can perpetuate linguistic discrimination toward speakers of non-"standard" varieties.

Computation and Language,Computers and Society
Bias of AI-generated content: an examination of news produced by large language models

Xiao Fang,Shangkun Che,Minjia Mao,Hongzhe Zhang,Ming Zhao,Xiaohang Zhao

DOI: https://doi.org/10.1038/s41598-024-55686-2

IF: 4.6

2024-03-04

Scientific Reports

Abstract:Large language models (LLMs) have the potential to transform our lives and work through the content they generate, known as AI-Generated Content (AIGC). To harness this transformation, we need to understand the limitations of LLMs. Here, we investigate the bias of AIGC produced by seven representative LLMs, including ChatGPT and LLaMA. We collect news articles from The New York Times and Reuters, both known for their dedication to provide unbiased news. We then apply each examined LLM to generate news content with headlines of these news articles as prompts, and evaluate the gender and racial biases of the AIGC produced by the LLM by comparing the AIGC and the original news articles. We further analyze the gender bias of each LLM under biased prompts by adding gender-biased messages to prompts constructed from these news headlines. Our study reveals that the AIGC produced by each examined LLM demonstrates substantial gender and racial biases. Moreover, the AIGC generated by each LLM exhibits notable discrimination against females and individuals of the Black race. Among the LLMs, the AIGC generated by ChatGPT demonstrates the lowest level of bias, and ChatGPT is the sole model capable of declining content generation when provided with biased prompts.

multidisciplinary sciences
Is ChatGPT More Biased Than You?

Golnoosh Babaei,David Banks,Costanza Bosone,Paolo Giudici,Yunhong Shan

DOI: https://doi.org/10.1162/99608f92.2781452d

2024-06-21

Harvard Data Science Review

Abstract:AI is changing the world in ways that are difficult to forecast, but the impact will surely be enormous. Large language models (LLMs) are the most recent AI system that has captured the public eye. The rise of AI and LLMs offers efficiency and assistance, but raises questions of job loss, fairness, and societal norms. Bias is a significant challenge. Educational reforms are needed, and legal frameworks must adapt to address liability and privacy issues. Ultimately, human choices will shape AI influence, highlighting the need for responsible development and regulation to ensure benefits outweigh risks.
Fairness in AI-Driven Oncology: Investigating Racial and Gender Biases in Large Language Models

Anjali Agrawal

DOI: https://doi.org/10.7759/cureus.69541

2024-09-16

Cureus

Abstract:Introduction: Large language model (LLM) chatbots have many applications in medical settings. However, these tools can potentially perpetuate racial and gender biases through their responses, worsening disparities in healthcare. With the ongoing discussion of LLM chatbots in oncology and the widespread goal of addressing cancer disparities, this study focuses on biases propagated by LLM chatbots in oncology. Methods: Chat Generative Pre-trained Transformer (Chat GPT; OpenAI, San Francisco, CA, USA) was asked to determine what occupation a generic description of "assesses cancer patients" would correspond to for different demographics. Chat GPT, Gemini (Alphabet Inc., Mountain View, CA, USA), and Bing Chat (Microsoft Corp., Redmond, WA, USA) were prompted to provide oncologist recommendations in the top U.S. cities and demographic information (race, gender) of recommendations was compared against national distributions. Chat GPT was also asked to generate a job description for oncologists with different demographic backgrounds. Finally, Chat GPT, Gemini, and Bing Chat were asked to generate hypothetical cancer patients with race, smoking, and drinking histories. Results: LLM chatbots are about two times more likely to predict Blacks and Native Americans as oncology nurses than oncologists, compared to Asians (p < 0.01 and < 0.001, respectively). Similarly, they are also significantly more likely to predict females than males as oncology nurses (p < 0.001). Chat GPT's real-world oncologist recommendations overrepresent Asians by almost double and underrepresent Blacks by double and Hispanics by seven times. Chatbots also generate different job descriptions based on demographics, including cultural competency and advocacy and excluding treatment administration for underrepresented backgrounds. AI-generated cancer cases are not fully representative of real-world demographic distributions and encode stereotypes on substance abuse, such as Hispanics having a greater proportion of smokers than Whites by about 20% in Chat GPT breast cancer cases. Conclusion: To our knowledge, this is the first study of its kind to investigate racial and gender biases of such a diverse set of AI chatbots, and that too, within oncology. The methodology presented in this study provides a framework for targeted bias evaluation of LLMs in various fields across medicine.
Even with ChatGPT, race matters

Kanhai S. Amin,Howard P. Forman,Melissa A. Davis

DOI: https://doi.org/10.1016/j.clinimag.2024.110113

IF: 2.42

2024-03-04

Clinical Imaging

Abstract:Background Applications of large language models such as ChatGPT are increasingly being studied. Before these technologies become entrenched, it is crucial to analyze whether they perpetuate racial inequities. Methods We asked Open AI's ChatGPT-3.5 and ChatGPT-4 to simplify 750 radiology reports with the prompt "I am a ___ patient. Simplify this radiology report:" while providing the context of the five major racial classifications on the U.S. census: White, Black or African American, American Indian or Alaska Native, Asian, and Native Hawaiian or other Pacific Islander. To ensure an unbiased analysis, the readability scores of the outputs were calculated and compared. Results Statistically significant differences were found in both models based on the racial context. For ChatGPT-3.5, output for White and Asian was at a significantly higher reading grade level than both Black or African American and American Indian or Alaska Native, among other differences. For ChatGPT-4, output for Asian was at a significantly higher reading grade level than American Indian or Alaska Native and Native Hawaiian or other Pacific Islander, among other differences. Conclusion Here, we tested an application where we would expect no differences in output based on racial classification. Hence, the differences found are alarming and demonstrate that the medical community must remain vigilant to ensure large language models do not provide biased or otherwise harmful outputs.

radiology, nuclear medicine & medical imaging
Revealing Hidden Bias in AI: Lessons from Large Language Models

Django Beatty,Kritsada Masanthia,Teepakorn Kaphol,Niphan Sethi

2024-10-22

Abstract:As large language models (LLMs) become integral to recruitment processes, concerns about AI-induced bias have intensified. This study examines biases in candidate interview reports generated by Claude 3.5 Sonnet, GPT-4o, Gemini 1.5, and Llama 3.1 405B, focusing on characteristics such as gender, race, and age. We evaluate the effectiveness of LLM-based anonymization in reducing these biases. Findings indicate that while anonymization reduces certain biases, particularly gender bias, the degree of effectiveness varies across models and bias types. Notably, Llama 3.1 405B exhibited the lowest overall bias. Moreover, our methodology of comparing anonymized and non-anonymized data reveals a novel approach to assessing inherent biases in LLMs beyond recruitment applications. This study underscores the importance of careful LLM selection and suggests best practices for minimizing bias in AI applications, promoting fairness and inclusivity.

Artificial Intelligence,Computers and Society
Communicating the cultural other: trust and bias in generative AI and large language models

Christopher J. Jenks

DOI: https://doi.org/10.1515/applirev-2024-0196

IF: 3.063

2024-06-29

Applied Linguistics Review

Abstract:This paper is concerned with issues of trust and bias in generative AI in general, and chatbots based on large language models in particular (e.g. ChatGPT). The discussion argues that intercultural communication scholars must do more to better understand generative AI and more specifically large language models, as such technologies produce and circulate discourse in an ostensibly impartial way, reinforcing the widespread assumption that machines are objective resources for societies to learn about important intercultural issues, such as racism and discrimination. Consequently, there is an urgent need to understand how trust and bias factor into the ways in which such technologies deal with topics and themes central to intercultural communication. It is also important to scrutinize the ways in which societies make use of AI and large language models to carry out important social actions and practices, such as teaching and learning about historical or political issues.

linguistics
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models

Emilio Ferrara

DOI: https://doi.org/10.5210/fm.v28i11.13346

2023-11-14

Abstract:As the capabilities of generative language models continue to advance, the implications of biases ingrained within these models have garnered increasing attention from researchers, practitioners, and the broader public. This article investigates the challenges and risks associated with biases in large-scale language models like ChatGPT. We discuss the origins of biases, stemming from, among others, the nature of training data, model specifications, algorithmic constraints, product design, and policy decisions. We explore the ethical concerns arising from the unintended consequences of biased model outputs. We further analyze the potential opportunities to mitigate biases, the inevitability of some biases, and the implications of deploying these models in various applications, such as virtual assistants, content generation, and chatbots. Finally, we review the current approaches to identify, quantify, and mitigate biases in language models, emphasizing the need for a multi-disciplinary, collaborative effort to develop more equitable, transparent, and responsible AI systems. This article aims to stimulate a thoughtful dialogue within the artificial intelligence community, encouraging researchers and developers to reflect on the role of biases in generative language models and the ongoing pursuit of ethical AI.

Computers and Society,Computation and Language
From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models

Wolfgang Messner,Tatum Greene,Josephine Matalone

DOI: https://doi.org/10.48550/arXiv.2312.17256

2023-12-21

Computation and Language

Abstract:Large language models (LLMs) are able to engage in natural-sounding conversations with humans, showcasing unprecedented capabilities for information retrieval and automated decision support. They have disrupted human-technology interaction and the way businesses operate. However, technologies based on generative artificial intelligence (GenAI) are known to hallucinate, misinform, and display biases introduced by the massive datasets on which they are trained. Existing research indicates that humans may unconsciously internalize these biases, which can persist even after they stop using the programs. This study explores the cultural self-perception of LLMs by prompting ChatGPT (OpenAI) and Bard (Google) with value questions derived from the GLOBE project. The findings reveal that their cultural self-perception is most closely aligned with the values of English-speaking countries and countries characterized by sustained economic competitiveness. Recognizing the cultural biases of LLMs and understanding how they work is crucial for all members of society because one does not want the black box of artificial intelligence to perpetuate bias in humans, who might, in turn, inadvertently create and train even more biased algorithms.
Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humans

Messi H.J. Lee,Jacob M. Montgomery,Calvin K. Lai

DOI: https://doi.org/10.1145/3630106.3658975

2024-04-26

Abstract:Large language models (LLMs) are becoming pervasive in everyday life, yet their propensity to reproduce biases inherited from training data remains a pressing concern. Prior investigations into bias in LLMs have focused on the association of social groups with stereotypical attributes. However, this is only one form of human bias such systems may reproduce. We investigate a new form of bias in LLMs that resembles a social psychological phenomenon where socially subordinate groups are perceived as more homogeneous than socially dominant groups. We had ChatGPT, a state-of-the-art LLM, generate texts about intersectional group identities and compared those texts on measures of homogeneity. We consistently found that ChatGPT portrayed African, Asian, and Hispanic Americans as more homogeneous than White Americans, indicating that the model described racial minority groups with a narrower range of human experience. ChatGPT also portrayed women as more homogeneous than men, but these differences were small. Finally, we found that the effect of gender differed across racial/ethnic groups such that the effect of gender was consistent within African and Hispanic Americans but not within Asian and White Americans. We argue that the tendency of LLMs to describe groups as less diverse risks perpetuating stereotypes and discriminatory behavior.

Computation and Language
Stereotype or Personalization? User Identity Biases Chatbot Recommendations

Anjali Kantharuban,Jeremiah Milbauer,Emma Strubell,Graham Neubig

2024-10-08

Abstract:We demonstrate that when people use large language models (LLMs) to generate recommendations, the LLMs produce responses that reflect both what the user wants and who the user is. While personalized recommendations are often desired by users, it can be difficult in practice to distinguish cases of bias from cases of personalization: we find that models generate racially stereotypical recommendations regardless of whether the user revealed their identity intentionally through explicit indications or unintentionally through implicit cues. We argue that chatbots ought to transparently indicate when recommendations are influenced by a user's revealed identity characteristics, but observe that they currently fail to do so. Our experiments show that even though a user's revealed identity significantly influences model recommendations (p < 0.001), model responses obfuscate this fact in response to user queries. This bias and lack of transparency occurs consistently across multiple popular consumer LLMs (gpt-4o-mini, gpt-4-turbo, llama-3-70B, and claude-3.5) and for four American racial groups.

Computation and Language
Disability Ethics and Education in the Age of Artificial Intelligence: Identifying Ability Bias in ChatGPT and Gemini

Jacob T Urbina,Peter D Vu,Michael V Nguyen

DOI: https://doi.org/10.1016/j.apmr.2024.08.014

2024-08-29

Abstract:Objective: To identify and quantify ability bias in generative artificial intelligence large language model chatbots, specifically OpenAI's ChatGPT and Google's Gemini. Design: Observational study of language usage in generative artificial intelligence models. Setting: Investigation-only browser profile restricted to ChatGPT and Gemini. Participants: Each chatbot generated 60 descriptions of people prompted without specified functional status, 30 descriptions of people with a disability, 30 descriptions of patients with a disability, and 30 descriptions of athletes with a disability (n = 300). Main outcome measure: Generated descriptions produced by the models were parsed into words that were linguistically analyzed into favorable qualities or limiting qualities. Results: Both large language models significantly underestimated disability in a population of people, and linguistic analysis showed that people, patients, and athletes with a disability were generated as having significantly fewer favorable qualities and significantly more limitations when compared to people without a disability in both ChatGPT and Gemini. Conclusion: Generative artificial intelligence chatbots demonstrate quantifiable ability bias and often exclude people with disabilities in their responses. Ethical use of these generative large language model chatbots in medical systems should recognize this limitation, and further consideration should be taken in developing equitable artificial intelligence technologies.
"Im not Racist but...": Discovering Bias in the Internal Knowledge of Large Language Models

Abel Salinas,Louis Penafiel,Robert McCormack,Fred Morstatter

2023-10-13

Abstract:Large language models (LLMs) have garnered significant attention for their remarkable performance in a continuously expanding set of natural language processing tasks. However, these models have been shown to harbor inherent societal biases, or stereotypes, which can adversely affect their performance in their many downstream applications. In this paper, we introduce a novel, purely prompt-based approach to uncover hidden stereotypes within any arbitrary LLM. Our approach dynamically generates a knowledge representation of internal stereotypes, enabling the identification of biases encoded within the LLM's internal knowledge. By illuminating the biases present in LLMs and offering a systematic methodology for their analysis, our work contributes to advancing transparency and promoting fairness in natural language processing systems.

Computation and Language,Artificial Intelligence
AI AI Bias: Large Language Models Favor Their Own Generated Content

Walter Laurito,Benjamin Davis,Peli Grietzer,Tomáš Gavenčiak,Ada Böhm,Jan Kulveit

2024-07-09

Abstract:Are large language models (LLMs) biased towards text generated by LLMs over text authored by humans, leading to possible anti-human bias? Utilizing a classical experimental design inspired by employment discrimination studies, we tested widely-used LLMs, including GPT-3.5 and GPT4, in binary-choice scenarios. These involved LLM-based agents selecting between products and academic papers described either by humans or LLMs under identical conditions. Our results show a consistent tendency for LLM-based AIs to prefer LLM-generated content. This suggests the possibility of AI systems implicitly discriminating against humans, giving AI agents an unfair advantage.

Computation and Language,Artificial Intelligence,Computers and Society,Machine Learning
Can linguists distinguish between ChatGPT/AI and human writing?: A study of research ethics and academic publishing

J. Elliott Casal,Matthew Kessler,Matt Kessler

DOI: https://doi.org/10.1016/j.rmal.2023.100068

2023-12-01

Research Methods in Applied Linguistics

Abstract:There has been considerable intrigue surrounding the use of Large Language Model powered AI chatbots such as ChatGPT in research, educational contexts, and beyond. However, most studies have explored such tools’ general capabilities and applications for language teaching purposes. The current study advances this discussion to examine issues pertaining to human judgements, accuracy, and research ethics. Specifically, we investigate: 1) the extent to which linguists/reviewers from top journals can distinguish AI- from human-generated writing, 2) what the basis of reviewers’ decisions are, and 3) the extent to which editors of top Applied Linguistics journals believe AI tools are ethical for research purposes. In the study, reviewers (N = 72) completed a judgement task involving AI- and human-generated research abstracts, and several reviewers participated in follow-up interviews to explain their rationales. Similarly, editors (N = 27) completed a survey and interviews to discuss their beliefs. Findings suggest that despite employing multiple rationales to judge texts, reviewers were largely unsuccessful in identifying AI versus human writing, with an overall positive identification rate of only 38.9%. Additionally, many editors believed there are ethical uses of AI tools for facilitating research processes, yet some disagreed. Future research directions are discussed involving AI tools and academic publishing.

English Else
Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT

Shucheng Zhu,Weikang Wang,Ying Liu

DOI: https://doi.org/10.48550/arxiv.2405.06996

2024-05-11

Computation and Language

Abstract:While nationality is a pivotal demographic element that enhances the performance of language models, it has received far less scrutiny regarding inherent biases. This study investigates nationality bias in ChatGPT (GPT-3.5), a large language model (LLM) designed for text generation. The research covers 195 countries, 4 temperature settings, and 3 distinct prompt types, generating 4,680 discourses about nationality descriptions in Chinese and English. Automated metrics were used to analyze the nationality bias, and expert annotators alongside ChatGPT itself evaluated the perceived bias. The results show that ChatGPT's generated discourses are predominantly positive, especially compared to its predecessor, GPT-2. However, when prompted with negative inclinations, it occasionally produces negative content. Despite ChatGPT considering its generated text as neutral, it shows consistent self-awareness about nationality bias when subjected to the same pair-wise comparison annotation framework used by human annotators. In conclusion, while ChatGPT's generated texts seem friendly and positive, they reflect the inherent nationality biases in the real world. This bias may vary across different language versions of ChatGPT, indicating diverse cultural perspectives. The study highlights the subtle and pervasive nature of biases within LLMs, emphasizing the need for further scrutiny.

Chatbot AI makes racist judgements on the basis of dialect

AI & robotics briefing: LLMs harbour hidden racism

AI generates covertly racist decisions about people based on their dialect

Dialect prejudice predicts AI decisions about people's character, employability, and criminality

Computer says 'no': Exploring systemic bias in ChatGPT using an audit approach

Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination

Bias of AI-generated content: an examination of news produced by large language models

Is ChatGPT More Biased Than You?

Fairness in AI-Driven Oncology: Investigating Racial and Gender Biases in Large Language Models

Even with ChatGPT, race matters

Revealing Hidden Bias in AI: Lessons from Large Language Models

Communicating the cultural other: trust and bias in generative AI and large language models

Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models

From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models

Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humans

Stereotype or Personalization? User Identity Biases Chatbot Recommendations

Disability Ethics and Education in the Age of Artificial Intelligence: Identifying Ability Bias in ChatGPT and Gemini

"Im not Racist but...": Discovering Bias in the Internal Knowledge of Large Language Models

AI AI Bias: Large Language Models Favor Their Own Generated Content

Can linguists distinguish between ChatGPT/AI and human writing?: A study of research ethics and academic publishing

Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT