Abstract:Recently, with the applications of algorithms in various risky scenarios, algorithmic fairness has been a serious concern and received lots of interest in machine learning community. In this paper, we focus on the bipartite ranking scenario, where the instances come from either the positive or negative class and the goal is to learn a ranking function that ranks positive instances higher than negative ones. We are interested in whether the learned ranking function can cause systematic disparity across different protected groups defined by sensitive attributes. While there could be a trade-off between fairness and performance, we propose a model agnostic post-processing framework xOrder for achieving fairness in bipartite ranking and maintaining the algorithm classification performance. In particular, we optimize a weighted sum of the utility as identifying an optimal warping path across different protected groups and solve it through a dynamic programming process. xOrder is compatible with various classification models and ranking fairness metrics, including supervised and unsupervised fairness metrics. In addition to binary groups, xOrder can be applied to multiple protected groups. We evaluate our proposed algorithm on four benchmark data sets and two real-world patient electronic health record repositories. xOrder consistently achieves a better balance between the algorithm utility and ranking fairness on a variety of datasets with different metrics. From the visualization of the calibrated ranking scores, xOrder mitigates the score distribution shifts of different groups compared with baselines. Moreover, additional analytical results verify that xOrder achieves a robust performance when faced with fewer samples and a bigger difference between training and testing ranking score distributions. For reproducing our results, we also made our source codes available at https://github.com/cuis15/xorder.

Rate-Optimal Rank Aggregation with Private Pairwise Rankings

Ranking Differential Privacy

The Application of Differential Privacy for Rank Aggregation: Privacy and Accuracy

Privacy-Preserving Fair Item Ranking

Rankmbpr: Rank-Aware Mutual Bayesian Personalized Ranking For Item Recommendationl

Analyzing Preference Data with Local Privacy: Optimal Utility and Enhanced Robustness

Towards Model-Agnostic Post-Hoc Adjustment for Balancing Ranking Fairness and Algorithm Utility

Sequential Manipulation Against Rank Aggregation: Theory and Algorithm

Bipartite Ranking Fairness through a Model Agnostic Ordering Adjustment

Pairwise Learning for Personalized Ranking with Noisy Comparisons

Fairness in Ranking: Robustness through Randomization without the Protected Attribute

Fairness in Recommendation Ranking through Pairwise Comparisons

Asymptotically Optimal Sequential Design for Rank Aggregation

Pareto Pairwise Ranking for Fairness Enhancement of Recommender Systems

Privacy Profiles for Private Selection

Instance-optimal Mean Estimation Under Differential Privacy

Bias-aware ranking from pairwise comparisons

Xorder: A Model Agnostic Post-Processing Framework for Achieving Ranking Fairness While Maintaining Algorithm Utility.

Collecting Preference Rankings under Local Differential Privacy

Fair Pairs: Fairness-Aware Ranking Recovery from Pairwise Comparisons