Abstract:Computer vision technologies have been applied to an increasingly wide range of applications from autonomous car navigation, to medical image analysis, to precision agriculture. Despite many of these exciting innovations, recent studies reveal a number of risks in using existing computer vision systems, suggesting results of such systems may be unfair or untrustworthy. For example, major commercial facial analysis tools were shown to have substantial accuracy disparities for people of different gender or with different skin colors (Buolamwini and Gebru 2018). Visual semantic role labeling models were found to exhibit societal biases and stereotypes (Zhao et al. 2017) such as frequently associating certain activity labels with specific gender (e.g., associate “cooking” with woman). Even worse, seemly accurate image classifiers may in fact made the predictions by picking up spurious correlations between objects and irrelevant background information rather than identifying meaningful features of the objects (Ribeiro, Singh, and Guestrin 2016). Many of the risks embedded in modern computer vision systems can be attributed to the use of a training dataset that is biased. Indeed, the computer vision community has long recognized that many visual datasets present varying degrees of build-in bias due to factors such as photographic style of photographers and selection from dataset curators (Torralba, Efros, and others 2011). Using these biased datasets to train machine learning models for addressing different computer vision tasks naturally leads to the phenomenon of “bias in, bias out” and results in undesirable performance. Thus, to mitigate the fairness, accountability, and transparency concerns in computer vision, a crucial step is to start the entire pipeline with high-quality visual datasets that, at least, are authentic representations of the visual world. In other words, being able to detect potential biases hidden in the datasets prior to model development is a key step in guarding against unfair or untrustworthy outcomes in computer vision. While a few techniques have been developed to automatically detect dataset biases (Tramer et al. 2017), the nonstructured nature of visual data makes bias discovery in image datasets particularly challenging. This is because no human-comprehensive attributes can be directly leveraged

My Model is Unfair, Do People Even Care? Visual Design Affects Trust and Perceived Bias in Machine Learning

Discovering Biases in Image Datasets with the Crowd

Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors

Understanding Bias in Machine Learning

On the Fairness of Machine-Assisted Human Decisions

Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairness

Analyzing Fairness of Computer Vision and Natural Language Processing Models

Integrating Behavioral, Economic, and Technical Insights to Understand and Address Algorithmic Bias: A Human-Centric Perspective

Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment

Designing for Confidence: The Impact of Visualizing Artificial Intelligence Decisions

Dispersion vs Disparity: Hiding Variability Can Encourage Stereotyping When Visualizing Social Outcomes

Towards Fair Machine Learning Software: Understanding and Addressing Model Bias Through Counterfactual Thinking

FairLay-ML: Intuitive Remedies for Unfairness in Data-Driven Social-Critical Algorithms

fairmodels: A Flexible Tool For Bias Detection, Visualization, And Mitigation

One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisions

The Equity Framework: Fairness Beyond Equalized Predictive Outcomes

Trust Your Gut: Comparing Human and Machine Inference from Noisy Visualizations

Communicating and combating algorithmic bias: effects of data diversity, labeler diversity, performance bias, and user feedback on AI trust

Modeling Techniques for Machine Learning Fairness: A Survey

Bias-Aware Design for Informed Decisions: Raising Awareness of Self-Selection Bias in User Ratings and Reviews

People Perceive Algorithmic Assessments as Less Fair and Trustworthy Than Identical Human Assessments