Causal Context Connects Counterfactual Fairness to Robust Prediction and Group Fairness

Jacy Reese Anthis,Victor Veitch
2023-10-31
Abstract:Counterfactual fairness requires that a person would have been classified in the same way by an AI or other algorithmic system if they had a different protected class, such as a different race or gender. This is an intuitive standard, as reflected in the U.S. legal system, but its use is limited because counterfactuals cannot be directly observed in real-world data. On the other hand, group fairness metrics (e.g., demographic parity or equalized odds) are less intuitive but more readily observed. In this paper, we use $\textit{causal context}$ to bridge the gaps between counterfactual fairness, robust prediction, and group fairness. First, we motivate counterfactual fairness by showing that there is not necessarily a fundamental trade-off between fairness and accuracy because, under plausible conditions, the counterfactually fair predictor is in fact accuracy-optimal in an unbiased target distribution. Second, we develop a correspondence between the causal graph of the data-generating process and which, if any, group fairness metrics are equivalent to counterfactual fairness. Third, we show that in three common fairness contexts$\unicode{x2013}$measurement error, selection on label, and selection on predictors$\unicode{x2013}$counterfactual fairness is equivalent to demographic parity, equalized odds, and calibration, respectively. Counterfactual fairness can sometimes be tested by measuring relatively simple group fairness metrics.
Machine Learning,Artificial Intelligence,Computers and Society
What problem does this paper attempt to address?
The problems that this paper attempts to solve mainly focus on the field of algorithm fairness, especially on how to link counterfactual fairness with group fairness and robust prediction. Specifically, the paper explores the following aspects: 1. **Motivation for counterfactual fairness**: The paper first provides a new motivation to explain counterfactual fairness from the perspective of robust prediction. By showing that under certain causal structures, counterfactual - fair predictors are optimal in accuracy in an unbiased target distribution, it is proved that counterfactual fairness is not only an ethical standard but also a technically reasonable goal. 2. **The relationship between counterfactual fairness and group fairness**: The second focus of the paper is to establish the correspondence between counterfactual fairness and group fairness metrics. Through causal graph analysis, the author determines which group fairness metrics can be equivalent to counterfactual fairness. This part solves the problem that counterfactual fairness is difficult to directly observe because counterfactual information cannot be directly obtained in the real world. 3. **Equivalence in specific situations**: The paper further shows that in three common fairness situations (measurement error, label selection, predictor variable selection), counterfactual fairness is respectively equivalent to demographic parity, equalized odds, and calibration. This indicates that in these specific situations, counterfactual fairness can be tested by measuring simple group fairness metrics. ### Formula representation - **Definition of counterfactual fairness**: \[ f(X(a)) = f(X(a')) \] where \(X(a)\) and \(X(a')\) respectively represent the potential outcomes of an individual under different protected classes \(a\) and \(a'\). - **Definition of pure chance association**: \[ Y\perp X\mid X_{\perp A}, A \] that is, given the protected class \(A\), the label \(Y\) only depends on the input part \(X_{\perp A}\) that is not affected by the protected class. - **Equivalence conditions between counterfactual fairness and group fairness**: - **Demographic parity**: \[ \text{Counterfactual fairness} \iff \text{There are no unblocked paths from } X_{\perp A} \text{ to } A \] - **Equalized odds**: \[ \text{Counterfactual fairness} \iff \text{All paths from } X_{\perp A} \text{ to } A \text{ are either blocked by variables other than } Y \text{ or contain } Y \] - **Calibration**: \[ \text{Counterfactual fairness} \iff \text{All paths from } Y \text{ to } A \text{ are either blocked by variables other than } X_{\perp A} \text{ or contain } X_{\perp A} \] ### Conclusion Through the above research, the paper provides a new theoretical basis for counterfactual fairness and shows how to achieve counterfactual fairness through group fairness metrics in practical applications. This is of great significance for the research and practice of algorithm fairness, especially for applications in high - risk fields such as medical treatment, recruitment, and financial lending.