Abstract:Recent literature has demonstrated that humans high in self-efficacy tend to resist AI advice despite its superior capabilities. In this study, we question the validity of this finding for a prevalent practice, in which AI capability is dynamically augmented and humans accumulate knowledge and interaction experience with AI. As such, humans high in self-efficacy, compared with those low in self-efficacy, are more likely to recognize AI capability augmentation due to their strong learning ability. We thus posit that this phenomenon, together with the AI's superior capability compared with that of humans, would invoke individuals' active cognitive reasoning by comparing themselves with respect to the AI system. Consequently, they tend to change their attitude to follow AI. We leverage a unique natural experiment of AI capability augmentation in the context of an on-demand food delivery service to verify our conjecture. We obtain several interesting findings: (1) Human riders high in self-efficacy indeed become more compliant with AI advice in the presence of an AI capability augmentation, in both the short and long run. However, (2) riders' short-term food delivery performance significantly decreases. Further mechanism analysis reveals that (3) such decline mainly occurs during an adaptation (learning) window when riders shift their decision-making objective for the delivery sequence from their own to the one recommended by the AI system; in the long run, their delivery performance eventually recovers to a high level. By contrast, (4) riders low in self-efficacy are trivially affected by AI capability augmentation. From a dynamic perspective, our findings shed light on the critical role of cognitive reasoning triggered by high human self-efficacy in human-AI collaboration, which prompts humans to better leverage the wisdom of AI and their own to achieve better collaborative decision-making outcomes.

Human Learning about AI Performance

Human-AI Learning Performance in Multi-Armed Bandits

Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction

How to Measure Human-AI Prediction Accuracy in Explainable AI Systems

Raising the Stakes: Performance Pressure Improves AI-Assisted Decision Making

Towards Understanding AI Delegation: The Role of Self-Efficacy and Visual Processing Ability

Human-AI Collaboration Increases Skill Tagging Speed but Degrades Accuracy

Capturing Humans' Mental Models of AI: An Item Response Theory Approach

AI Knowledge: Improving AI Delegation through Human Enablement

Do Humans Trust Advice More if it Comes from AI? An Analysis of Human-AI Interactions

On the Perception of Difficulty: Differences between Humans and AI

Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making

Navigating the Manager-AI Divide

Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models

Promise or Peril? When Human Efficacy Meets AI Capability Augmentation

Heuristic Reasoning in AI: Instrumental Use and Mimetic Absorption

Towards Effective Human-AI Decision-Making: The Role of Human Learning in Appropriate Reliance on AI Advice

Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn

Improving Human-AI Collaboration With Descriptions of AI Behavior

Can We Distinguish Machine Learning from Human Learning?