Abstract:Problem definition: Data analytics models and machine learning algorithms are increasingly deployed to support consequential decision-making processes, from deciding which applicants will receive job offers and loans to university enrollments and medical interventions. However, recent studies show these models may unintentionally amplify human bias and yield significant unfavorable decisions to specific groups. Methodology/results: We propose a distributionally robust classification model with a fairness constraint that encourages the classifier to be fair in the equality of opportunity criterion. We use a type-[Formula: see text] Wasserstein ambiguity set centered at the empirical distribution to represent distributional uncertainty and derive a conservative reformulation for the worst-case equal opportunity unfairness measure. We show that the model is equivalent to a mixed binary conic optimization problem, which standard off-the-shelf solvers can solve. We propose a convex, hinge-loss-based model for large problem instances whose reformulation does not incur binary variables to improve scalability. Moreover, we also consider the distributionally robust learning problem with a generic ground transportation cost to hedge against the label and sensitive attribute uncertainties. We numerically examine the performance of our proposed models on five real-world data sets related to individual analysis. Compared with the state-of-the-art methods, our proposed approaches significantly improve fairness with negligible loss of predictive accuracy in the testing data set. Managerial implications: Our paper raises awareness that bias may arise when predictive models are used in service and operations. It generally comes from human bias, for example, imbalanced data collection or low sample sizes, and is further amplified by algorithms. Incorporating fairness constraints and the distributionally robust optimization (DRO) scheme is a powerful way to alleviate algorithmic biases. Funding: This work was supported by the National Science Foundation [Grants 2342505 and 2343869] and the Chinese University of Hong Kong [Grant 4055191]. Supplemental Material: The online appendices are available at https://doi.org/10.1287/msom.2022.0230 .

Optimal data-driven hiring with equity for underrepresented groups

Don't let Ricci v. DeStefano Hold You Back: A Bias-Aware Legal Solution to the Hiring Paradox

Learning Optimal Fair Policies

What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring

Fairness and Bias in Algorithmic Hiring: a Multidisciplinary Survey

Learning Fair Policies for Multi-stage Selection Problems from Observational Data

Modelling the long-term fairness dynamics of data-driven targeted help on job seekers

Does Fair Ranking Improve Minority Outcomes? Understanding the Interplay of Human and Algorithmic Biases in Online Hiring

Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search

Towards Fairness and Privacy: A Novel Data Pre-processing Optimization Framework for Non-binary Protected Attributes

Fairness in AI-Driven Recruitment: Challenges, Metrics, Methods, and Future Directions

A Short-term Intervention for Long-term Fairness in the Labor Market

AI for hiring in context: a perspective on overcoming the unique challenges of employment research to mitigate disparate impact

Fairness of recommender systems in the recruitment domain: an analysis from technical and legal perspectives

Optimisation Strategies for Ensuring Fairness in Machine Learning: With and Without Demographics

Wasserstein Robust Classification with Fairness Constraints

Estimating and Implementing Conventional Fairness Metrics With Probabilistic Protected Features

Adaptive Sampling Strategies to Construct Equitable Training Datasets

Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making

Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination

Hiring Fairly in the Age of Algorithms