The (Im)possibility of fairness

Sorelle A. Friedler,Carlos Scheidegger,Suresh Venkatasubramanian

DOI: https://doi.org/10.1145/3433949

IF: 22.7

2021-04-01

Communications of the ACM

Abstract:What does it mean to be fair?

computer science, theory & methods, software engineering, hardware & architecture

What problem does this paper attempt to address?

Escaping the Impossibility of Fairness: From Formal to Substantive Algorithmic Fairness

Ben Green

DOI: https://doi.org/10.1007/s13347-022-00584-6

2023-02-26

Abstract:Efforts to promote equitable public policy with algorithms appear to be fundamentally constrained by the "impossibility of fairness" (an incompatibility between mathematical definitions of fairness). This technical limitation raises a central question about algorithmic fairness: How can computer scientists and policymakers support equitable policy reforms with algorithms? In this article, I argue that promoting justice with algorithms requires reforming the methodology of algorithmic fairness. First, I diagnose the problems of the current methodology for algorithmic fairness, which I call "formal algorithmic fairness." Because formal algorithmic fairness restricts analysis to isolated decision-making procedures, it leads to the impossibility of fairness and to models that exacerbate oppression despite appearing "fair." Second, I draw on theories of substantive equality from law and philosophy to propose an alternative methodology, which I call "substantive algorithmic fairness." Because substantive algorithmic fairness takes a more expansive scope of analysis, it enables an escape from the impossibility of fairness and provides a rigorous guide for alleviating injustice with algorithms. In sum, substantive algorithmic fairness presents a new direction for algorithmic fairness: away from formal mathematical models of "fair" decision-making and toward substantive evaluations of whether and how algorithms can promote justice in practice.

Computers and Society,Machine Learning
Fairness definitions explained

Sahil Verma,Julia Rubin

DOI: https://doi.org/10.1145/3194770.3194776

2018-05-29

Abstract:Algorithm fairness has started to attract the attention of researchers in AI, Software Engineering and Law communities, with more than twenty different notions of fairness proposed in the last few years. Yet, there is no clear agreement on which definition to apply in each situation. Moreover, the detailed differences between multiple definitions are difficult to grasp. To address this issue, this paper collects the most prominent definitions of fairness for the algorithmic classification problem, explains the rationale behind these definitions, and demonstrates each of them on a single unifying case-study. Our analysis intuitively explains why the same case can be considered fair according to some definitions and unfair according to others.
The Measure and Mismeasure of Fairness

Sam Corbett-Davies,Johann D. Gaebler,Hamed Nilforoshan,Ravi Shroff,Sharad Goel

2023-08-15

Abstract:The field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last decade, several formal, mathematical definitions of fairness have gained prominence. Here we first assemble and categorize these definitions into two broad families: (1) those that constrain the effects of decisions on disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions typically result in strongly Pareto dominated decision policies. For example, in the case of college admissions, adhering to popular formal conceptions of fairness would simultaneously result in lower student-body diversity and a less academically prepared class, relative to what one could achieve by explicitly tailoring admissions policies to achieve desired outcomes. In this sense, requiring that these fairness definitions hold can, perversely, harm the very groups they were designed to protect. In contrast to axiomatic notions of fairness, we argue that the equitable design of algorithms requires grappling with their context-specific consequences, akin to the equitable design of policy. We conclude by listing several open challenges in fair machine learning and offering strategies to ensure algorithms are better aligned with policy goals.

Computers and Society
Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning

Megha Srivastava,Hoda Heidari,Andreas Krause

DOI: https://doi.org/10.48550/arXiv.1902.04783

2019-12-08

Abstract:Fairness for Machine Learning has received considerable attention, recently. Various mathematical formulations of fairness have been proposed, and it has been shown that it is impossible to satisfy all of them simultaneously. The literature so far has dealt with these impossibility results by quantifying the tradeoffs between different formulations of fairness. Our work takes a different perspective on this issue. Rather than requiring all notions of fairness to (partially) hold at the same time, we ask which one of them is the most appropriate given the societal domain in which the decision-making model is to be deployed. We take a descriptive approach and set out to identify the notion of fairness that best captures \emph{lay people's perception of fairness}. We run adaptive experiments designed to pinpoint the most compatible notion of fairness with each participant's choices through a small number of tests. Perhaps surprisingly, we find that the most simplistic mathematical definition of fairness---namely, demographic parity---most closely matches people's idea of fairness in two distinct application scenarios. This conclusion remains intact even when we explicitly tell the participants about the alternative, more complicated definitions of fairness, and we reduce the cognitive burden of evaluating those notions for them. Our findings have important implications for the Fair ML literature and the discourse on formalizing algorithmic fairness.

Computers and Society
Harm Ratio: A Novel and Versatile Fairness Criterion

Soroush Ebadian,Rupert Freeman,Nisarg Shah

2024-10-04

Abstract:Envy-freeness has become the cornerstone of fair division research. In settings where each individual is allocated a disjoint share of collective resources, it is a compelling fairness axiom which demands that no individual strictly prefer the allocation of another individual to their own. Unfortunately, in many real-life collective decision-making problems, the goal is to choose a (common) public outcome that is equally applicable to all individuals, and the notion of envy becomes vacuous. Consequently, this literature has avoided studying fairness criteria that focus on individuals feeling a sense of jealousy or resentment towards other individuals (rather than towards the system), missing out on a key aspect of fairness. In this work, we propose a novel fairness criterion, individual harm ratio, which is inspired by envy-freeness but applies to a broad range of collective decision-making settings. Theoretically, we identify minimal conditions under which this criterion and its groupwise extensions can be guaranteed, and study the computational complexity of related problems. Empirically, we conduct experiments with real data to show that our fairness criterion is powerful enough to differentiate between prominent decision-making algorithms for a range of tasks from voting and fair division to participatory budgeting and peer review.

Computer Science and Game Theory,Artificial Intelligence
What Is Fairness? On the Role of Protected Attributes and Fictitious Worlds

Ludwig Bothmann,Kristina Peters,Bernd Bischl

2024-06-03

Abstract:A growing body of literature in fairness-aware machine learning (fairML) aims to mitigate machine learning (ML)-related unfairness in automated decision-making (ADM) by defining metrics that measure fairness of an ML model and by proposing methods to ensure that trained ML models achieve low scores on these metrics. However, the underlying concept of fairness, i.e., the question of what fairness is, is rarely discussed, leaving a significant gap between centuries of philosophical discussion and the recent adoption of the concept in the ML community. In this work, we try to bridge this gap by formalizing a consistent concept of fairness and by translating the philosophical considerations into a formal framework for the training and evaluation of ML models in ADM systems. We argue that fairness problems can arise even without the presence of protected attributes (PAs), and point out that fairness and predictive performance are not irreconcilable opposites, but that the latter is necessary to achieve the former. Furthermore, we argue why and how causal considerations are necessary when assessing fairness in the presence of PAs by proposing a fictitious, normatively desired (FiND) world in which PAs have no causal effects. In practice, this FiND world must be approximated by a warped world in which the causal effects of the PAs are removed from the real-world data. Finally, we achieve greater linguistic clarity in the discussion of fairML. We outline algorithms for practical applications and present illustrative experiments on COMPAS data.

Machine Learning,Artificial Intelligence,Computers and Society
The Fairness Field Guide: Perspectives from Social and Formal Sciences

Alycia N. Carey,Xintao Wu

DOI: https://doi.org/10.48550/arXiv.2201.05216

2022-03-09

Abstract:Over the past several years, a slew of different methods to measure the fairness of a machine learning model have been proposed. However, despite the growing number of publications and implementations, there is still a critical lack of literature that explains the interplay of fair machine learning with the social sciences of philosophy, sociology, and law. We hope to remedy this issue by accumulating and expounding upon the thoughts and discussions of fair machine learning produced by both social and formal (specifically machine learning and statistics) sciences in this field guide. Specifically, in addition to giving the mathematical and algorithmic backgrounds of several popular statistical and causal-based fair machine learning methods, we explain the underlying philosophical and legal thoughts that support them. Further, we explore several criticisms of the current approaches to fair machine learning from sociological and philosophical viewpoints. It is our hope that this field guide will help fair machine learning practitioners better understand how their algorithms align with important humanistic values (such as fairness) and how we can, as a field, design methods and metrics to better serve oppressed and marginalized populaces.

Artificial Intelligence,Computers and Society,Machine Learning
The Flawed Foundations of Fair Machine Learning

Robert Lee Poe,Soumia Zohra El Mestari

2023-06-02

Abstract:The definition and implementation of fairness in automated decisions has been extensively studied by the research community. Yet, there hides fallacious reasoning, misleading assertions, and questionable practices at the foundations of the current fair machine learning paradigm. Those flaws are the result of a failure to understand that the trade-off between statistically accurate outcomes and group similar outcomes exists as independent, external constraint rather than as a subjective manifestation as has been commonly argued. First, we explain that there is only one conception of fairness present in the fair machine learning literature: group similarity of outcomes based on a sensitive attribute where the similarity benefits an underprivileged group. Second, we show that there is, in fact, a trade-off between statistically accurate outcomes and group similar outcomes in any data setting where group disparities exist, and that the trade-off presents an existential threat to the equitable, fair machine learning approach. Third, we introduce a proof-of-concept evaluation to aid researchers and designers in understanding the relationship between statistically accurate outcomes and group similar outcomes. Finally, suggestions for future work aimed at data scientists, legal scholars, and data ethicists that utilize the conceptual and experimental framework described throughout this article are provided.

Computers and Society,Machine Learning
What’s Fair in International Politics? Equity, Equality, and Foreign Policy Attitudes

Kathleen E. Powers,Joshua D. Kertzer,Deborah J. Brooks,Stephen G. Brooks

DOI: https://doi.org/10.1177/00220027211041393

2021-09-29

Journal of Conflict Resolution

Abstract:How do concerns about fairness shape foreign policy preferences? In this article, we show that fairness has two faces—one concerning equity, the other concerning equality—and that taking both into account can shed light on the structure of important foreign policy debates. Fielding an original survey on a national sample of Americans, we show that different types of Americans think about fairness in different ways, and that these fairness concerns shape foreign policy preferences: individuals who emphasize equity are far more sensitive to concerns about burden sharing, are far less likely to support US involvement abroad when other countries aren’t paying their fair share, and often support systematically different foreign policies than individuals who emphasize equality. As long as IR scholars focus only on the equality dimension of fairness, we miss much about how fairness concerns matter in world politics.

political science,international relations
Algorithmic Fairness: Choices, Assumptions, and Definitions

Shira Mitchell,Eric Potash,Solon Barocas,Alexander D'Amour,Kristian Lum

DOI: https://doi.org/10.1146/annurev-statistics-042720-125902

IF: 7.9

2021-03-07

Annual Review of Statistics and Its Application

Abstract:A recent wave of research has attempted to define fairness quantitatively. In particular, this work has explored what fairness might mean in the context of decisions based on the predictions of statistical and machine learning models. The rapid growth of this new field has led to wildly inconsistent motivations, terminology, and notation, presenting a serious challenge for cataloging and comparing definitions. This article attempts to bring much-needed order. First, we explicate the various choices and assumptions made—often implicitly—to justify the use of prediction-based decision-making. Next, we show how such choices and assumptions can raise fairness concerns and we present a notationally consistent catalog of fairness definitions from the literature. In doing so, we offer a concise reference for thinking through the choices, assumptions, and fairness considerations of prediction-based decision-making.

statistics & probability,mathematics, interdisciplinary applications
Fairness views and cooperation under varying levels of economic inequality

Wasilios Hariskos,Jakob Neitzel,Lauri Sääksvuori

DOI: https://doi.org/10.1371/journal.pone.0288790

IF: 3.7

2023-11-09

PLoS ONE

Abstract:This paper investigates the impact of economic inequality on people's perceptions of fairness and willingness to cooperate. Using experimental and survey data, we distinguish people's injunctive perceptions of fairness from experimentally observed behavioral patterns. We find that impartial observers hold shared perceptions of fair contribution rules. Individuals with their own money at stake hold conflicting views over fair contribution rules. We find that contribution patterns are more scattered under strong inequality than under weak inequality. Overall, we observe that voluntary contributions are lower under strong inequality than under weak inequality. Our results contribute to the debate about the behavioral consequences of income and wealth inequalities in modern societies.
Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches

Lindsay Weinberg

DOI: https://doi.org/10.1613/jair.1.13196

2022-05-06

Abstract:This survey article assesses and compares existing critiques of current fairness-enhancing technical interventions into machine learning (ML) that draw from a range of non-computing disciplines, including philosophy, feminist studies, critical race and ethnic studies, legal studies, anthropology, and science and technology studies. It bridges epistemic divides in order to offer an interdisciplinary understanding of the possibilities and limits of hegemonic computational approaches to ML fairness for producing just outcomes for society's most marginalized. The article is organized according to nine major themes of critique wherein these different fields intersect: 1) how "fairness" in AI fairness research gets defined; 2) how problems for AI systems to address get formulated; 3) the impacts of abstraction on how AI tools function and its propensity to lead to technological solutionism; 4) how racial classification operates within AI fairness research; 5) the use of AI fairness measures to avoid regulation and engage in ethics washing; 6) an absence of participatory design and democratic deliberation in AI fairness considerations; 7) data collection practices that entrench "bias," are non-consensual, and lack transparency; 8) the predatory inclusion of marginalized groups into AI systems; and 9) a lack of engagement with AI's long-term social and ethical outcomes. Drawing from these critiques, the article concludes by imagining future ML fairness research directions that actively disrupt entrenched power dynamics and structural injustices in society.

Machine Learning,Artificial Intelligence,Computers and Society
The Fairness Fair: Bringing Human Perception into Collective Decision-Making

Hadi Hosseini

DOI: https://doi.org/10.48550/arXiv.2312.14402

2023-12-22

Abstract:Fairness is one of the most desirable societal principles in collective decision-making. It has been extensively studied in the past decades for its axiomatic properties and has received substantial attention from the multiagent systems community in recent years for its theoretical and computational aspects in algorithmic decision-making. However, these studies are often not sufficiently rich to capture the intricacies of human perception of fairness in the ambivalent nature of the real-world problems. We argue that not only fair solutions should be deemed desirable by social planners (designers), but they should be governed by human and societal cognition, consider perceived outcomes based on human judgement, and be verifiable. We discuss how achieving this goal requires a broad transdisciplinary approach ranging from computing and AI to behavioral economics and human-AI interaction. In doing so, we identify shortcomings and long-term challenges of the current literature of fair division, describe recent efforts in addressing them, and more importantly, highlight a series of open research directions.

Artificial Intelligence,Computer Science and Game Theory,Multiagent Systems,Theoretical Economics
The Unfairness of $\varepsilon$-Fairness

Tolulope Fadina,Thorsten Schmidt

2024-06-18

Abstract:Fairness in decision-making processes is often quantified using probabilistic metrics. However, these metrics may not fully capture the real-world consequences of unfairness. In this article, we adopt a utility-based approach to more accurately measure the real-world impacts of decision-making process. In particular, we show that if the concept of $\varepsilon$-fairness is employed, it can possibly lead to outcomes that are maximally unfair in the real-world context. Additionally, we address the common issue of unavailable data on false negatives by proposing a reduced setting that still captures essential fairness considerations. We illustrate our findings with two real-world examples: college admissions and credit risk assessment. Our analysis reveals that while traditional probability-based evaluations might suggest fairness, a utility-based approach uncovers the necessary actions to truly achieve equality. For instance, in the college admission case, we find that enhancing completion rates is crucial for ensuring fairness. Summarizing, this paper highlights the importance of considering the real-world context when evaluating fairness.

Machine Learning,Theoretical Economics,Mathematical Finance
The inversion paradox, and classification of fairness notions

Uriel Feige

DOI: https://doi.org/10.48550/arXiv.2311.16770

2023-11-28

Computer Science and Game Theory

Abstract:Several different fairness notions have been introduced in the context of fair allocation of goods. In this manuscript, we compare between some fairness notions that are used in settings in which agents have arbitrary (perhaps unequal) entitlements to the goods. This includes the proportional share, the anyprice share, the weighted maximin share, weighted envy freeness, maximum weight Nash social welfare and competitive equilibrium. We perform this comparison in two settings, that of a divisible homogeneous good and arbitrary valuations, and that of indivisible goods and additive valuations. Different fairness notions are not always compatible with each other, and might dictate selecting different allocations. The purpose of our work is to clarify various properties of fairness notions, so as to allow, when needed, to make an educated choice among them. Also, such a study may motivate introducing new fairness notions, or modifications to existing fairness notions. Among other properties, we introduce definitions for monotonicity that postulate that having higher entitlement should be better to the agent than having lower entitlement. Some monotonicity notions, such as population monotonicity and weight monotonicity, appeared in previous work, but we prefer to consider other monotonicity properties that we refer to as global monotonicity and individual monotonicity. We find that some of the fairness notions (but not all) violate our monotonicity notions in a strong sense, that we refer to as the inversion paradox. Under this paradox, a fairness notion enforces that the value received by an agent decreases when the entitlement of the agent increases.
Group Fairness Is Not Derivable From Justice: a Mathematical Proof

Nicolò Cangiotti,Michele Loi

DOI: https://doi.org/10.48550/arXiv.2202.03880

2022-02-08

Abstract:We argue that an imperfect criminal law procedure cannot be group-fair, if 'group fairness' involves ensuring the same chances of acquittal or convictions to all innocent defendants independently of their morally arbitrary features. We show mathematically that only a perfect procedure (involving no mistake), a non-deterministic one, or a degenerate one (everyone or no one is convicted) can guarantee group fairness, in the general case. Following a recent proposal, we adopt a definition of group fairness, requiring that individuals who are equal in merit ought to have the same statistical chances of obtaining advantages and disadvantages, in a way that is statistically independent of any of their feature that does not count as merit. We explain by mathematical argument that the only imperfect procedures offering an a-priori guarantee of fairness in relation to all non-merit trait are lotteries or degenerate ones (i.e., everyone or no one is convicted). To provide a more intuitive point of view, we exploit an adjustment of the well-known ROC space, in order to represent all possible procedures in our model by a schematic diagram. The argument seems to be equally valid for all human procedures, provided they are imperfect. This clearly includes algorithmic decision-making, including decisions based on statistical predictions, since in practice all statistical models are error prone.

Computers and Society
On the apparent conflict between individual and group fairness

Reuben Binns

DOI: https://doi.org/10.1145/3351095.3372864

2020-01-22

Abstract:A distinction has been drawn in fair machine learning research between 'group' and 'individual' fairness measures. Many technical research papers assume that both are important, but conflicting, and propose ways to minimise the trade-offs between these measures. This paper argues that this apparent conflict is based on a misconception. It draws on discussions from within the fair machine learning research, and from political and legal philosophy, to argue that individual and group fairness are not fundamentally in conflict. First, it outlines accounts of egalitarian fairness which encompass plausible motivations for both group and individual fairness, thereby suggesting that there need be no conflict in principle. Second, it considers the concept of individual justice, from legal philosophy and jurisprudence, which seems similar but actually contradicts the notion of individual fairness as proposed in the fair machine learning literature. The conclusion is that the apparent conflict between individual and group fairness is more of an artefact of the blunt application of fairness measures, rather than a matter of conflicting principles. In practice, this conflict may be resolved by a nuanced consideration of the sources of 'unfairness' in a particular deployment context, and the carefully justified application of measures to mitigate it.
The Myth of Complete AI-Fairness

Virginia Dignum

DOI: https://doi.org/10.48550/arXiv.2104.12544

2021-04-06

Computers and Society

Abstract:The idea of fairness and justice has long and deep roots in Western civilization, and is strongly linked to ethics. It is therefore not strange that it is core to the current discussion about the ethics of development and use of AI systems. In this short paper, I wish to further motivate my position in this matter: ``I will never be completely fair. Nothing ever is. The point is not complete fairness, but the need to establish metrics and thresholds for fairness that ensure trust in AI systems".
Subjective fairness in algorithmic decision-support

Sarra Tajouri,Alexis Tsoukiàs

2024-06-28

Abstract:The treatment of fairness in decision-making literature usually involves quantifying fairness using objective measures. This work takes a critical stance to highlight the limitations of these approaches (group fairness and individual fairness) using sociological insights. First, we expose how these metrics often fail to reflect societal realities. By neglecting crucial historical, cultural, and social factors, they fall short of capturing all discriminatory practices. Second, we redefine fairness as a subjective property moving from a top-down to a bottom-up approach. This shift allows the inclusion of diverse stakeholders perceptions, recognizing that fairness is not merely about objective metrics but also about individuals views on their treatment. Finally, we aim to use explanations as a mean to achieve fairness. Our approach employs explainable clustering to form groups based on individuals subjective perceptions to ensure that individuals who see themselves as similar receive similar treatment. We emphasize the role of explanations in achieving fairness, focusing not only on procedural fairness but also on providing subjective explanations to convince stakeholders of their fair treatment.

Computers and Society
Fairness As A Constraint

Jiancai Pi

DOI: https://doi.org/10.1108/08288660810899386

2008-01-01

Humanomics

Abstract:Purpose - The purpose of this short paper is to provide an alternative method to explain the reason why the outcome of the ultimatum game based on the well-known self-interest model is not feasible in real life.Design/methodology/approach - Contrary to traditional models which treat fairness as a variable of the utility function, the paper takes fairness as a constraint to the utility function. The problem is solved through the standard incentive theory and bargaining theory.Findings - It was found that fairness constraints play an important role in determining the eventual outcome of the ultimatum game.Research limitations/implications - The paper's approach may set up a useful framework for incorporating fairness into economics, and bridge the gaps between the fairness approach and the standard incentive theory and bargaining theory.Practical implications - When tackling the problem of distribution based on fairness, the paper shows that the cultural background and special situation should be taken seriously.Originality/value - In this paper, the concept of the fairness constraint is introduced into a formal framework.

The (Im)possibility of fairness

Escaping the Impossibility of Fairness: From Formal to Substantive Algorithmic Fairness

Fairness definitions explained

The Measure and Mismeasure of Fairness

Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learning

Harm Ratio: A Novel and Versatile Fairness Criterion

What Is Fairness? On the Role of Protected Attributes and Fictitious Worlds

The Fairness Field Guide: Perspectives from Social and Formal Sciences

The Flawed Foundations of Fair Machine Learning

What’s Fair in International Politics? Equity, Equality, and Foreign Policy Attitudes

Algorithmic Fairness: Choices, Assumptions, and Definitions

Fairness views and cooperation under varying levels of economic inequality

Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches

The Fairness Fair: Bringing Human Perception into Collective Decision-Making

The Unfairness of $\varepsilon$-Fairness

The inversion paradox, and classification of fairness notions

Group Fairness Is Not Derivable From Justice: a Mathematical Proof

On the apparent conflict between individual and group fairness

The Myth of Complete AI-Fairness

Subjective fairness in algorithmic decision-support

Fairness As A Constraint