Abstract:Objective: This study manipulates the presence and reliability of AI recommendations for risky decisions to measure the effect on task performance, behavioral consequences of trust, and deviation from a probability matching collaborative decision-making model. Background: Although AI decision support improves performance, people tend to underutilize AI recommendations, particularly when outcomes are uncertain. As AI reliability increases, task performance improves, largely due to higher rates of compliance (following action recommendations) and reliance (following no-action recommendations). Methods: In a between-subject design, participants were assigned to a high reliability AI, low reliability AI, or a control condition. Participants decided whether to bet that their team would win in a series of basketball games tying compensation to performance. We evaluated task performance (in accuracy and signal detection terms) and the behavioral consequences of trust (via compliance and reliance). Results: AI recommendations improved task performance, had limited impact on risk-taking behavior, and were under-valued by participants. Accuracy, sensitivity (d'), and reliance increased in the high reliability AI condition, but there was no effect on response bias (c) or compliance. Participant behavior was only consistent with a probability matching model for compliance in the low reliability condition. Conclusion: In a pay-off structure that incentivized risk-taking, the primary value of the AI recommendations was in determining when to perform no action (i.e., pass on bets). Application: In risky contexts, designers need to consider whether action or no-action recommendations will be more influential to design appropriate interventions.

When Should a Decision Maker Ignore the Advice of a Decision Aid?

Learning When to Advise Human Decision Makers

Decision-aid or Controller? Steering Human Decision Makers with Algorithms

The Role of Heuristics and Biases During Complex Choices with an AI Teammate

Knowing When to Pass: The Effect of AI Reliability in Risky Decision Contexts

On the (non-) reliance on algorithms—A decision-theoretic account

Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making

Algorithms in selection decisions: Effective, but unappreciated

Boosting Human Decision-making with AI-Generated Decision Aids

Algorithmic Assistance with Recommendation-Dependent Preferences

Humans rely more on algorithms than social influence as a task becomes more difficult

Decision Making "Biases" and Support for Assumption-Based Higher-Order Reasoning

Learning to Make Adherence-Aware Advice

Correcting User Decisions Based on Incorrect Machine Learning Decisions

The Best Decisions Are Not the Best Advice: Making Adherence-Aware Recommendations

Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks

Persuasion, Delegation, and Private Information in Algorithm-Assisted Decisions

Technological Shocks and Algorithmic Decision Aids in Credence Goods Markets

Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making

Dynamics of Reliance on Algorithmic Advice

Does AI help humans make better decisions? A statistical evaluation framework for experimental and observational studies