Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making

Shuai Ma,Ying Lei,Xinru Wang,Chengbo Zheng,Chuhan Shi,Ming Yin,Xiaojuan Ma

DOI: https://doi.org/10.48550/arXiv.2301.05809

2023-01-14

Abstract:In AI-assisted decision-making, it is critical for human decision-makers to know when to trust AI and when to trust themselves. However, prior studies calibrated human trust only based on AI confidence indicating AI's correctness likelihood (CL) but ignored humans' CL, hindering optimal team decision-making. To mitigate this gap, we proposed to promote humans' appropriate trust based on the CL of both sides at a task-instance level. We first modeled humans' CL by approximating their decision-making models and computing their potential performance in similar instances. We demonstrated the feasibility and effectiveness of our model via two preliminary studies. Then, we proposed three CL exploitation strategies to calibrate users' trust explicitly/implicitly in the AI-assisted decision-making process. Results from a between-subjects experiment (N=293) showed that our CL exploitation strategies promoted more appropriate human trust in AI, compared with only using AI confidence. We further provided practical implications for more human-compatible AI-assisted decision-making.

Human-Computer Interaction,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in AI - assisted decision - making, how can human decision - makers trust AI at appropriate times and trust themselves at appropriate times. Specifically, the paper points out that the existing research mainly calibrates human trust based on the confidence level of AI, but ignores the human's own Correctness Likelihood (CL), which leads to sub - optimal team decision - making effects. Therefore, the paper proposes a new framework, aiming to promote appropriate trust by considering the CL of both AI and humans simultaneously, thereby enhancing the complementary performance of human - machine collaboration.

The core problems of the paper are:
- How can we promote the appropriate trust of human decision - makers in AI during the AI - assisted decision - making process by combining the Correctness Likelihood (CL) of AI and humans?
- When the CL of AI is low (high) and the CL of humans is even lower (higher), should humans be encouraged to question (trust) AI?

To answer these questions, the paper proposes a framework that not only considers the CL of AI, but also estimates the CL of humans in each task instance, and calibrates human trust according to the relative CL of both parties. The paper verifies the feasibility and effectiveness of the method through two preliminary studies, and calibrates users' trust explicitly or implicitly through three CL utilization strategies, and finally verifies the effectiveness of these strategies through a large - scale experiment (N = 293).