Abstract:When a model's performance differs across socially or culturally relevant groups--like race, gender, or the intersections of many such groups--it is often called "biased." While much of the work in algorithmic fairness over the last several years has focused on developing various definitions of model fairness (the absence of group-wise model performance disparities) and eliminating such "bias," much less work has gone into rigorously measuring it. In practice, it important to have high quality, human digestible measures of model performance disparities and associated uncertainty quantification about them that can serve as inputs into multi-faceted decision-making processes. In this paper, we show both mathematically and through simulation that many of the metrics used to measure group-wise model performance disparities are themselves statistically biased estimators of the underlying quantities they purport to represent. We argue that this can cause misleading conclusions about the relative group-wise model performance disparities along different dimensions, especially in cases where some sensitive variables consist of categories with few members. We propose the "double-corrected" variance estimator, which provides unbiased estimates and uncertainty quantification of the variance of model performance across groups. It is conceptually simple and easily implementable without statistical software package or numerical optimization. We demonstrate the utility of this approach through simulation and show on a real dataset that while statistically biased estimators of group-wise model performance disparities indicate statistically significant differences, when accounting for statistical bias in the estimator, the estimated between-group disparities are no longer statistically significant.

Observing Context Improves Disparity Estimation when Race is Unobserved

Fairness under Unawareness: Assessing Disparity when Protected Class is Unobserved.

Estimating Racial Disparities When Race is Not Observed

Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination

A Causal Framework for Observational Studies of Discrimination

Auditing and Enforcing Conditional Fairness via Optimal Transport

Fair Visual Recognition via Intervention with Proxy Features

Consistency, efficiency and robustness of conditional disparity methods

Fair and Effective Policing for Neighborhood Safety: Understanding and Overcoming Selection Biases

De-biasing "bias" measurement

De-Biasing the Bias: Methods for Improving Disparity Assessments with Noisy Group Measurements

Mitigating Included- and Omitted-Variable Bias in Estimates of Disparate Impact

Regression with race-modifiers: towards equity and interpretability

De-Biasing Models of Biased Decisions: A Comparison of Methods Using Mortgage Application Data

Debiasing Alternative Data for Credit Underwriting Using Causal Inference

Contextual Optimizer through Neighborhood Estimation for prescriptive analysis

Estimating and Implementing Conventional Fairness Metrics With Probabilistic Protected Features

COBIAS: Contextual Reliability in Bias Assessment

Causal Context Connects Counterfactual Fairness to Robust Prediction and Group Fairness

Privacy-Preserving Race/Ethnicity Estimation for Algorithmic Bias Measurement in the U.S

The Target Study: A Conceptual Model and Framework for Measuring Disparity