Abstract:Computer vision technologies have been applied to an increasingly wide range of applications from autonomous car navigation, to medical image analysis, to precision agriculture. Despite many of these exciting innovations, recent studies reveal a number of risks in using existing computer vision systems, suggesting results of such systems may be unfair or untrustworthy. For example, major commercial facial analysis tools were shown to have substantial accuracy disparities for people of different gender or with different skin colors (Buolamwini and Gebru 2018). Visual semantic role labeling models were found to exhibit societal biases and stereotypes (Zhao et al. 2017) such as frequently associating certain activity labels with specific gender (e.g., associate “cooking” with woman). Even worse, seemly accurate image classifiers may in fact made the predictions by picking up spurious correlations between objects and irrelevant background information rather than identifying meaningful features of the objects (Ribeiro, Singh, and Guestrin 2016). Many of the risks embedded in modern computer vision systems can be attributed to the use of a training dataset that is biased. Indeed, the computer vision community has long recognized that many visual datasets present varying degrees of build-in bias due to factors such as photographic style of photographers and selection from dataset curators (Torralba, Efros, and others 2011). Using these biased datasets to train machine learning models for addressing different computer vision tasks naturally leads to the phenomenon of “bias in, bias out” and results in undesirable performance. Thus, to mitigate the fairness, accountability, and transparency concerns in computer vision, a crucial step is to start the entire pipeline with high-quality visual datasets that, at least, are authentic representations of the visual world. In other words, being able to detect potential biases hidden in the datasets prior to model development is a key step in guarding against unfair or untrustworthy outcomes in computer vision. While a few techniques have been developed to automatically detect dataset biases (Tramer et al. 2017), the nonstructured nature of visual data makes bias discovery in image datasets particularly challenging. This is because no human-comprehensive attributes can be directly leveraged

AI image generators often give racist and sexist results: can they be fixed?

Bias in Generative AI

Discovering Biases in Image Datasets with the Crowd

AI-generated faces influence gender stereotypes and racial homogenization

AI can be sexist and racist — it’s time to make it fair

Gender and Ethnicity Bias of Text-to-Image Generative Artificial Intelligence in Medical Imaging, Part 1: Preliminary Evaluation

A Taxonomy of the Biases of the Images created by Generative Artificial Intelligence

AI & robotics briefing: Can AI's bias problem be fixed?

Beyond the stereotypes: Artificial Intelligence image generation and diversity in anesthesiology

The Gendered Lens of AI: Examining News Imagery Across Digital Spaces

New Job, New Gender? Measuring the Social Bias in Image Generation Models

Considering How Machine‐Learning Algorithms (Re)produce Social Biases in Generated Faces

Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation

Generative AI Carries Non-Democratic Biases and Stereotypes: Representation of Women, Black Individuals, Age Groups, and People with Disability in AI-Generated Images across Occupations

She Works, He Works: A Curious Exploration of Gender Bias in AI-Generated Imagery

Smiling Women Pitching Down: Auditing Representational and Presentational Gender Biases in Image Generative AI

Artificial Intelligence Portrayals in Orthopaedic Surgery

Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI

Detecting race and gender bias in visual representation of AI on web search engines

Biases in Generative Art -- A Causal Look from the Lens of Art History

Gender and Ethnicity Bias of Text-to-Image Generative Artificial Intelligence in Medical Imaging, Part 2: Analysis of DALL-E 3