Fair and green hyperparameter optimization via multi-objective and multiple information source Bayesian optimization

Antonio Candelieri,Andrea Ponti,Francesco Archetti

DOI: https://doi.org/10.1007/s10994-024-06515-0

IF: 5.414

2024-03-02

Machine Learning

Abstract:It has been recently remarked that focusing only on accuracy in searching for optimal Machine Learning models amplifies biases contained in the data, leading to unfair predictions and decision supports. Recently, multi-objective hyperparameter optimization has been proposed to search for Machine Learning models which offer equally Pareto-efficient trade-offs between accuracy and fairness. Although these approaches proved to be more versatile than fairness-aware Machine Learning algorithms—which instead optimize accuracy constrained to some threshold on fairness—their carbon footprint could be dramatic, due to the large amount of energy required in the case of large datasets. We propose an approach named FanG-HPO: fair and green hyperparameter optimization (HPO), based on both multi-objective and multiple information source Bayesian optimization. FanG-HPO uses subsets of the large dataset to obtain cheap approximations (aka information sources) of both accuracy and fairness, and multi-objective Bayesian optimization to efficiently identify Pareto-efficient (accurate and fair) Machine Learning models. Experiments consider four benchmark (fairness) datasets and four Machine Learning algorithms, and provide an assessment of FanG-HPO against both fairness-aware Machine Learning approaches and two state-of-the-art Bayesian optimization tools addressing multi-objective and energy-aware optimization.

computer science, artificial intelligence

What problem does this paper attempt to address?

This paper mainly discusses how to consider fairness and energy efficiency in the hyperparameter optimization process of machine learning (ML) models. Traditional optimization methods only focus on accuracy, which may amplify biases in the data and result in unfair predictions. In recent years, multi-objective hyperparameter optimization has been proposed to find models that achieve Pareto-optimal balance between accuracy and fairness. However, these methods may consume a large amount of energy when dealing with large datasets, causing environmental impacts.

The paper proposes a new approach called FanG-HPO (Fairness and Green Hyperparameter Optimization), which is based on multi-objective and multi-information source Bayesian optimization. FanG-HPO uses subsets of large datasets to obtain low-cost approximations of accuracy and fairness, and efficiently identifies ML models that are both accurate and fair through multi-objective Bayesian optimization. Experiments compare FanG-HPO with fairness-aware ML algorithms and two state-of-the-art Bayesian optimization tools that focus on multi-objective and energy-aware optimization, respectively.

The main contributions of the paper include:
1. Comparative analysis of fairness-aware ML and fairness-aware automated machine learning (AutoML) algorithms.
2. Proposal of a novel fairness and green hyperparameter optimization algorithm, FanG-HPO.
3. Computational evaluation of the performance of FanG-HPO compared to the other two state-of-the-art BO suites.

The paper also reviews related work in the fields of fairness, energy efficiency, and Bayesian optimization. FanG-HPO aims to provide AutoML practitioners with tools for developing more fair and green decision support systems to address the environmental impacts of climate change and artificial intelligence.