Abstract:Objective: This study manipulates the presence and reliability of AI recommendations for risky decisions to measure the effect on task performance, behavioral consequences of trust, and deviation from a probability matching collaborative decision-making model. Background: Although AI decision support improves performance, people tend to underutilize AI recommendations, particularly when outcomes are uncertain. As AI reliability increases, task performance improves, largely due to higher rates of compliance (following action recommendations) and reliance (following no-action recommendations). Methods: In a between-subject design, participants were assigned to a high reliability AI, low reliability AI, or a control condition. Participants decided whether to bet that their team would win in a series of basketball games tying compensation to performance. We evaluated task performance (in accuracy and signal detection terms) and the behavioral consequences of trust (via compliance and reliance). Results: AI recommendations improved task performance, had limited impact on risk-taking behavior, and were under-valued by participants. Accuracy, sensitivity (d'), and reliance increased in the high reliability AI condition, but there was no effect on response bias (c) or compliance. Participant behavior was only consistent with a probability matching model for compliance in the low reliability condition. Conclusion: In a pay-off structure that incentivized risk-taking, the primary value of the AI recommendations was in determining when to perform no action (i.e., pass on bets). Application: In risky contexts, designers need to consider whether action or no-action recommendations will be more influential to design appropriate interventions.

Understanding User Reliance on AI in Assisted Decision-Making

Designing for Appropriate Reliance: The Roles of AI Uncertainty Presentation, Initial User Decision, and User Demographics in AI-Assisted Decision-Making

A Decision Theoretic Framework for Measuring AI Reliance

"Are You Really Sure?" Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Making

Towards Effective Human-AI Decision-Making: The Role of Human Learning in Appropriate Reliance on AI Advice

" Are You Really Sure?" Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Making

Inferring Human Vision in a Human-Like Way: Key Factors Influencing the Cognitive Processing of Level-1 Visual Perspective-Taking

Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making

Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making

AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions

Modeling Human Trust and Reliance in AI-Assisted Decision Making: A Markovian Approach

A Survey of AI Reliance

Do Humans Trust Advice More if it Comes from AI? An Analysis of Human-AI Interactions

Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations

Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations

Dynamic Selection of Reliance Calibration Cues With AI Reliance Model

Knowing When to Pass: The Effect of AI Reliability in Risky Decision Contexts

Eyes on the Game: Deciphering Implicit Human Signals to Infer Human Proficiency, Trust, and Intent

To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems

Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance

Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making