Abstract:Recently, with the applications of algorithms in various risky scenarios, algorithmic fairness has been a serious concern and received lots of interest in machine learning community. In this paper, we focus on the bipartite ranking scenario, where the instances come from either the positive or negative class and the goal is to learn a ranking function that ranks positive instances higher than negative ones. We are interested in whether the learned ranking function can cause systematic disparity across different protected groups defined by sensitive attributes. While there could be a trade-off between fairness and performance, we propose a model agnostic post-processing framework xOrder for achieving fairness in bipartite ranking and maintaining the algorithm classification performance. In particular, we optimize a weighted sum of the utility as identifying an optimal warping path across different protected groups and solve it through a dynamic programming process. xOrder is compatible with various classification models and ranking fairness metrics, including supervised and unsupervised fairness metrics. In addition to binary groups, xOrder can be applied to multiple protected groups. We evaluate our proposed algorithm on four benchmark data sets and two real-world patient electronic health record repositories. xOrder consistently achieves a better balance between the algorithm utility and ranking fairness on a variety of datasets with different metrics. From the visualization of the calibrated ranking scores, xOrder mitigates the score distribution shifts of different groups compared with baselines. Moreover, additional analytical results verify that xOrder achieves a robust performance when faced with fewer samples and a bigger difference between training and testing ranking score distributions. For reproducing our results, we also made our source codes available at https://github.com/cuis15/xorder.

Multi-partite Ranking with Multi-Class AdaBoost Algorithm.

Instance-Ranking: A New Perspective to Consider the Instance Dependency for Classification

AdaBoost-based Multi-attribute Classification Technology and Its Application

Multi-class AdaBoost ELM

Rankmbpr: Rank-Aware Mutual Bayesian Personalized Ranking For Item Recommendationl

Two-Stage Multi-Class Adaboost for Facial Expression Recognition

Multiple Pairwise Ranking with Implicit Feedback.

Analysis of Regression Tree Fitting Algorithms in Learning to Rank

Position-Aware ListMLE: A Sequential Learning Process for Ranking.

Stock Ranking with Multi-Task Learning

Top-K Pairwise Ranking: Bridging the Gap Among Ranking-Based Measures for Multi-Label Classification

An Empirical Evaluation of Four Algorithms for Multi-Class Classification: Mart, ABC-Mart, Robust LogitBoost, and ABC-LogitBoost

AOSO-LogitBoost: Adaptive One-Vs-One LogitBoost for Multi-Class Problem

Learning to Collaborate: Multi-Scenario Ranking Via Multi-Agent Reinforcement Learning.

Pareto Pairwise Ranking for Fairness Enhancement of Recommender Systems

Multi-Scenario Ranking with Adaptive Feature Learning

Alpha-Beta Sampling for Pairwise Ranking in One-Class Collaborative Filtering

Ada-Ranker: A Data Distribution Adaptive Ranking Paradigm for Sequential Recommendation

Bipartite Ranking Fairness through a Model Agnostic Ordering Adjustment

Adaptively Learning to Select-Rank in Online Platforms

Totally-Corrective Multi-Class Boosting