Mitigating Biases in Collective Decision-Making: Enhancing Performance in the Face of Fake News

Axel Abels,Elias Fernandez Domingos,Ann Nowé,Tom Lenaerts

2024-03-11

Abstract:Individual and social biases undermine the effectiveness of human advisers by inducing judgment errors which can disadvantage protected groups. In this paper, we study the influence these biases can have in the pervasive problem of fake news by evaluating human participants' capacity to identify false headlines. By focusing on headlines involving sensitive characteristics, we gather a comprehensive dataset to explore how human responses are shaped by their biases. Our analysis reveals recurring individual biases and their permeation into collective decisions. We show that demographic factors, headline categories, and the manner in which information is presented significantly influence errors in human judgment. We then use our collected data as a benchmark problem on which we evaluate the efficacy of adaptive aggregation algorithms. In addition to their improved accuracy, our results highlight the interactions between the emergence of collective intelligence and the mitigation of participant biases.

Human-Computer Interaction,Machine Learning,Social and Information Networks

What problem does this paper attempt to address?

The paper attempts to address the issue of mitigating bias in collective decision-making processes, particularly in the application of fake news detection. Specifically, the study explores how individual and social biases influence human judgment by evaluating participants' ability to identify false headlines and proposes several algorithms to optimize the collective decision-making process, thereby improving decision accuracy and reducing participant bias.

The main objectives of the paper include:
1. **Understanding the impact of bias on collective decision-making**: By collecting data from human participants in identifying headlines related to sensitive groups, the study explores how individual biases permeate into collective decisions.
2. **Evaluating the effectiveness of different algorithms**: Using the collected data as benchmark problems, the study assesses the effectiveness of static (e.g., majority voting) and adaptive (e.g., EXP4, MetaCMAB, ExpertiseTree) aggregation algorithms in mitigating bias.
3. **Enhancing collective intelligence**: The paper demonstrates how advanced machine learning algorithms can improve the quality of collective decisions by adjusting aggregation strategies to mitigate participant bias. Specifically, it showcases how the ExpertiseTree algorithm customizes aggregation methods based on different sensitive groups to enhance overall performance.