Abstract:Machine learning (ML) is widely used to moderate online content. Despite its scalability relative to human moderation, the use of ML introduces unique challenges to content moderation. One such challenge is predictive multiplicity: multiple competing models for content classification may perform equally well on average, yet assign conflicting predictions to the same content. This multiplicity can result from seemingly innocuous choices during model development, such as random seed selection for parameter initialization. We experimentally demonstrate how content moderation tools can arbitrarily classify samples as toxic, leading to arbitrary restrictions on speech. We discuss these findings in terms of human rights set out by the International Covenant on Civil and Political Rights (ICCPR), namely freedom of expression, non-discrimination, and procedural justice. We analyze (i) the extent of predictive multiplicity among state-of-the-art LLMs used for detecting toxic content; (ii) the disparate impact of this arbitrariness across social groups; and (iii) how model multiplicity compares to unambiguous human classifications. Our findings indicate that the up-scaled algorithmic moderation risks legitimizing an algorithmic leviathan, where an algorithm disproportionately manages human rights. To mitigate such risks, our study underscores the need to identify and increase the transparency of arbitrariness in content moderation applications. Since algorithmic content moderation is being fueled by pressing social concerns, such as disinformation and hate speech, our discussion on harms raises concerns relevant to policy debates. Our findings also contribute to content moderation and intermediary liability laws being discussed and passed in many countries, such as the Digital Services Act in the European Union, the Online Safety Act in the United Kingdom, and the Fake News Bill in Brazil.

The Algorithmic Leviathan: Arbitrariness, Fairness, and Opportunity in Algorithmic Decision Making Systems

AI’s fairness problem: understanding wrongful discrimination in the context of automated decision-making

Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature

Algorithmic decision making and the cost of fairness

Fair and Unbiased Algorithmic Decision Making: Current State and Future Challenges

Machine Decisions and Human Consequences

Discrimination in the Age of Algorithms

Arbitrariness and Social Prediction: the Confounding Role of Variance in Fair Classification

Algorithmic Fairness and Statistical Discrimination

Designing Equitable Algorithms

Fairness in Algorithmic Recourse Through the Lens of Substantive Equality of Opportunity

Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction

Algorithms and Decision-Making in the Public Sector

Distributive Justice and Fairness Metrics in Automated Decision-making: How Much Overlap Is There?

Algorithms, Incentives, and Democracy

Algorithmic Arbitrariness in Content Moderation

Algorithmic Harms in Child Welfare: Uncertainties in Practice, Organization, and Street-level Decision-Making

Algorithmic Monoculture and Social Welfare

On the Fairness of Machine-Assisted Human Decisions

On the Long-term Impact of Algorithmic Decision Policies: Effort Unfairness and Feature Segregation through Social Learning

Procedural Fairness as Stepping Stone for Successful Implementation of Algorithmic Decision-Making in Public Administration: Review and Outlook