Fair Adversarial Gradient Tree Boosting

Vincent Grari,Boris Ruf,Sylvain Lamprier,Marcin Detyniecki

DOI: https://doi.org/10.48550/arXiv.1911.05369

2019-11-18

Abstract:Fair classification has become an important topic in machine learning research. While most bias mitigation strategies focus on neural networks, we noticed a lack of work on fair classifiers based on decision trees even though they have proven very efficient. In an up-to-date comparison of state-of-the-art classification algorithms in tabular data, tree boosting outperforms deep learning. For this reason, we have developed a novel approach of adversarial gradient tree boosting. The objective of the algorithm is to predict the output $Y$ with gradient tree boosting while minimizing the ability of an adversarial neural network to predict the sensitive attribute $S$. The approach incorporates at each iteration the gradient of the neural network directly in the gradient tree boosting. We empirically assess our approach on 4 popular data sets and compare against state-of-the-art algorithms. The results show that our algorithm achieves a higher accuracy while obtaining the same level of fairness, as measured using a set of different common fairness definitions.

Machine Learning,Artificial Intelligence,Computers and Society

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to achieve fair classification in machine - learning models, especially in decision - tree - based models. Although most existing bias - mitigation strategies focus on neural networks, the author notices that there is relatively little research on fair classifiers built on decision trees. Given the efficient performance of decision trees on tabular data, the paper proposes a new method - Adversarial Gradient Tree Boosting, which aims to predict the output \(Y\) while minimizing the ability of the adversarial neural network to predict the sensitive attribute \(S\). Specifically, this method achieves this goal by directly incorporating the gradients of the neural network into the gradient - boosting tree in each iteration.

The main contributions of the paper are as follows:
1. Propose the first adversarial - learning method applicable to general classifiers (including decision trees).
2. Apply adversarial learning to decision trees to achieve fair classification.
3. Experimentally verify the performance of the proposed method and its variants in comparison with the existing state - of - the - art methods under two different fairness metrics. The results show that this method can achieve the same level of fairness while maintaining high accuracy.

Through this method, the paper not only improves the prediction accuracy of the model, but also effectively reduces the bias in the model, making it more fair and reliable in practical applications.