Abstract:Artificial intelligence algorithms are increasingly adopted as decisional aides by public bodies, with the promise of overcoming biases of human decision-makers. At the same time, they may introduce new biases in the human-algorithm interaction. Drawing on psychology and public administration literatures, we investigate two key biases: overreliance on algorithmic advice even in the face of warning signals from other sources (automation bias), and selective adoption of algorithmic advice when this corresponds to stereotypes (selective adherence). We assess these via three experimental studies conducted in the NetherlandsWe discuss the implications of our findings for public sector decision making in the age of automation. Overall, our study speaks to potential negative effects of automation of the administrative state for already vulnerable and disadvantaged citizens.

What problem does this paper attempt to address?

The problem this paper attempts to address is how human decision-makers handle algorithm outputs when artificial intelligence algorithms are used as decision support tools in public sector decision-making, and whether this handling leads to specific cognitive biases. Specifically, the author focuses on two main biases: 1. **Automation Bias**: This refers to human decision-makers' over-reliance on algorithmic recommendations even when faced with warning signals or contradictory information from other sources. Decision-makers may uncritically accept algorithmic recommendations even if they contradict additional evidence. 2. **Selective Adherence**: This refers to decision-makers' tendency to adopt recommendations that align with their stereotypes about certain groups. For example, when an algorithm predicts that a member of a negatively stereotyped minority group is at high risk, decision-makers may be more willing to accept this recommendation. To explore these issues, the author designed three experimental studies conducted among Dutch citizens and civil servants. Through these experiments, the author aims to test the following hypotheses: - **H1**: When faced with similar external contradictory evidence, decision-makers are more likely to trust and follow algorithmic recommendations rather than human recommendations (automation bias). - **H2**: Decision-makers are more likely to follow recommendations that align with stereotypes about the decision subject (whether algorithmic or human recommendations) (selective adherence). - **H3**: Selective adherence may be more severe when decision-makers receive algorithmic recommendations rather than human recommendations (exacerbated selective adherence). These studies aim to reveal the potential cognitive biases and their impacts when using artificial intelligence algorithms as decision support tools in the public sector. This not only helps to understand the application of technology in public administration but also provides policymakers with suggestions to improve algorithm usage and reduce biases.

Human-AI Interactions in Public Sector Decision-Making: "Automation Bias" and "Selective Adherence" to Algorithmic Advice

When the Automated Fire Backfires: the Adoption of Algorithm-based HR Decision-making Could Induce Consumer's Unfavorable Ethicality Inferences of the Company

Michael is better than Mehmet: exploring the perils of algorithmic biases and selective adherence to advice from automated decision support systems in hiring

Leveraging Artificial Intelligence for Public Sector Decision-Making: Balancing Accountability and Efficiency in Digital Public Services

Bending the Automation Bias Curve: A Study of Human and AI-based Decision Making in National Security Contexts

In AI we trust? Perceptions about automated decision-making by artificial intelligence

"The Human Must Remain the Central Focus": Subjective Fairness Perceptions in Automated Decision-Making

Algorithms and Decision-Making in the Public Sector

Analysing and Organising Human Communications for AI Fairness-Related Decisions: Use Cases from the Public Sector

THE DARK SIDE OF ARTIFICIAL INTELLIGENCE ON THE BASIS OF PUBLIC ADMINISTRATION

Human-Machine Collaborative Decision-Making in Organizations: Examining the Impact of Algorithm Prediction Bias on Decision Bias and Perceived Fairness

Artificial Discretion as a Tool of Governance: A Framework for Understanding the Impact of Artificial Intelligence on Public Administration

The impact of AI errors in a human-in-the-loop process

The Challenges of AI-Decision-Making in Government and Administrative Law: A Proposal for Regulatory Design

Procedural Fairness as Stepping Stone for Successful Implementation of Algorithmic Decision-Making in Public Administration: Review and Outlook

Automating Automaticity: How the Context of Human Choice Affects the Extent of Algorithmic Bias

Governing by Algorithm? No Noise and (Potentially) Less Bias

Experimental Evaluation of Algorithm-Assisted Human Decision-Making: Application to Pretrial Public Safety Assessment

New Pythias of public administration: ambiguity and choice in AI systems as challenges for governance

Will Algorithms Blind People? The Effect of Explainable AI and Decision-Makers’ Experience on AI-supported Decision-Making in Government

Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector