Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning

Esther Rolf,Max Simchowitz,Sarah Dean,Lydia T. Liu,Daniel Björkegren,Moritz Hardt,Joshua Blumenstock
DOI: https://doi.org/10.48550/arXiv.2003.06740
2020-07-16
Abstract:While real-world decisions involve many competing objectives, algorithmic decisions are often evaluated with a single objective function. In this paper, we study algorithmic policies which explicitly trade off between a private objective (such as profit) and a public objective (such as social welfare). We analyze a natural class of policies which trace an empirical Pareto frontier based on learned scores, and focus on how such decisions can be made in noisy or data-limited regimes. Our theoretical results characterize the optimal strategies in this class, bound the Pareto errors due to inaccuracies in the scores, and show an equivalence between optimal strategies and a rich class of fairness-constrained profit-maximizing policies. We then present empirical results in two different contexts -- online content recommendation and sustainable abalone fisheries -- to underscore the applicability of our approach to a wide range of practical decisions. Taken together, these results shed light on inherent trade-offs in using machine learning for decisions that impact social welfare.
Machine Learning
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: In real - world decision - making, multiple competing goals are usually involved (for example, private interests such as profit and public interests such as social welfare), but existing algorithmic decisions are often evaluated based only on a single objective function. This may lead to some negative social impacts, for example, loans that maximize profit may have an adverse effect on borrowers, or false news may damage the democratic system. To solve this problem, the author proposes a new framework, aiming to balance these competing goals through machine - learning methods. Specifically, this research focuses on how to develop strategies that can balance private and public interests through learned scores in the presence of noise or limited data. The author analyzes a class of natural policies based on the empirical Pareto frontier and explores how to make decisions in such situations. ### The core problems of the paper are summarized as follows: 1. **Multi - objective optimization**: In many practical applications, decision - making needs to consider multiple objectives simultaneously (such as profit and social welfare). Traditional machine - learning models usually optimize only one objective, which may lead to other important objectives being ignored. 2. **The impact of noisy data**: In practical applications, the data used to evaluate different objectives may be inaccurate or limited. Therefore, how to effectively balance multiple objectives in the presence of noise is a challenge. 3. **Fairness and social welfare**: The author points out that existing definitions of fairness may not be able to fully solve broader social fairness issues. Therefore, they propose a new framework, aiming to promote social welfare by explicitly balancing multiple objectives. ### Overview of the solution: - **Score - based strategies**: The author proposes a class of score - based strategies. These scores can predict an individual's contribution to different objectives (such as the contribution to total welfare and profit). Through these scores, an empirical Pareto frontier can be constructed, thereby finding the optimal multi - objective strategy. - **Theoretical analysis**: The author provides theoretical results, describes the error bounds of the Pareto frontier in the case of inaccurate scores, and shows the equivalence between the optimal strategy and a class of profit - maximizing strategies subject to fairness constraints. - **Empirical research**: The author verifies the effectiveness of this framework through two different application scenarios (online content recommendation and sustainable abalone fishery), indicating that it can be applied in a variety of practical decision - making. In conclusion, this paper aims to solve the limitations of existing machine - learning models in handling complex, multi - objective decision - making, especially in cases involving social welfare, by introducing a multi - objective optimization framework.