Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making

Charvi Rastogi,Yunfeng Zhang,Dennis Wei,Kush R. Varshney,Amit Dhurandhar,Richard Tomsett

DOI: https://doi.org/10.48550/arXiv.2010.07938

2022-04-05

Abstract:Several strands of research have aimed to bridge the gap between artificial intelligence (AI) and human decision-makers in AI-assisted decision-making, where humans are the consumers of AI model predictions and the ultimate decision-makers in high-stakes applications. However, people's perception and understanding are often distorted by their cognitive biases, such as confirmation bias, anchoring bias, availability bias, to name a few. In this work, we use knowledge from the field of cognitive science to account for cognitive biases in the human-AI collaborative decision-making setting, and mitigate their negative effects on collaborative performance. To this end, we mathematically model cognitive biases and provide a general framework through which researchers and practitioners can understand the interplay between cognitive biases and human-AI accuracy. We then focus specifically on anchoring bias, a bias commonly encountered in human-AI collaboration. We implement a time-based de-anchoring strategy and conduct our first user experiment that validates its effectiveness in human-AI collaborative decision-making. With this result, we design a time allocation strategy for a resource-constrained setting that achieves optimal human-AI collaboration under some assumptions. We, then, conduct a second user experiment which shows that our time allocation strategy with explanation can effectively de-anchor the human and improve collaborative performance when the AI model has low confidence and is incorrect.

Human-Computer Interaction,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in artificial - intelligence - assisted decision - making, human decision - makers are affected by cognitive biases, especially anchoring bias, which causes them to make inaccurate or sub - optimal decisions when collaborating with AI. Specifically, the paper focuses on how to mitigate the negative impact of these cognitive biases on human - machine collaboration performance and improve the accuracy of human - machine collaboration by understanding and modeling these cognitive biases.

### Main contributions of the paper:
1. **Constructed the Biased Bayesian Framework**: The paper provides a framework based on Bayesian theory for modeling the artificial - intelligence - assisted decision - making process with cognitive biases. This framework incorporates some known cognitive biases (such as confirmation bias, availability bias, etc.) to better understand the role of these biases in human - machine collaboration.
2. **Studied Anchoring Bias**: The paper specifically focuses on anchoring bias, a common cognitive bias in human - machine collaboration, where human decision - makers overly rely on the initial prediction (anchor) provided by AI, thus limiting their ability to explore other hypotheses. The paper experimentally verified that allocating more time can help decision - makers break free from the anchor and reduce the impact of anchoring bias.
3. **Proposed a Time Allocation Strategy**: To maximize the accuracy of human - machine collaboration, the paper proposed a time allocation strategy that takes into account anchoring bias and changes in AI prediction accuracy. The paper demonstrated that under certain assumptions, this strategy can achieve optimal human - machine collaboration results.
4. **Empirical Research**: Through two sets of user experiments, the paper verified the effectiveness of the time allocation strategy. The experimental results showed that when AI predictions are inaccurate and have low confidence, displaying confidence information can help human decision - makers break free from incorrect AI predictions and improve collaboration performance.

### Main methods and techniques:
- **Bayesian Modeling**: The paper uses Bayesian theory to model the cognitive processes of human decision - makers, especially the decision - making process after introducing AI predictions. By adjusting the parameters of the Bayesian formula, the paper simulates the impact of different cognitive biases.
- **Experimental Design**: The paper designed two sets of user experiments. The first set of experiments verified the effect of time allocation on reducing anchoring bias, and the second set of experiments tested the effectiveness of the confidence - based time allocation strategy.
- **Mathematical Modeling**: The paper uses mathematical models (such as formulas (4) and (5)) to describe the impact of cognitive biases on the decision - making process. These models are helpful for understanding the specific mechanisms of different biases.

### Conclusion:
Through theoretical analysis and empirical research, the paper shows how to mitigate the impact of cognitive biases on human - machine collaboration through time and confidence information, thereby improving the accuracy and efficiency of decision - making. This is of great significance for the application of artificial - intelligence - assisted decision - making in high - risk fields (such as medicine, finance, and justice).