Abstract:Artificial Intelligence (AI) is increasingly used to build Decision Support Systems (DSS) across many domains. This paper describes a series of experiments designed to observe human response to different characteristics of a DSS such as accuracy and bias, particularly the extent to which participants rely on the DSS, and the performance they achieve. In our experiments, participants play a simple online game inspired by so-called "wildcat" (i.e., exploratory) drilling for oil. The landscape has two layers: a visible layer describing the costs (terrain), and a hidden layer describing the reward (oil yield). Participants in the control group play the game without receiving any assistance, while in treatment groups they are assisted by a DSS suggesting places to drill. For certain treatments, the DSS does not consider costs, but only rewards, which introduces a bias that is observable by users. Between subjects, we vary the accuracy and bias of the DSS, and observe the participants' total score, time to completion, the extent to which they follow or ignore suggestions. We also measure the acceptability of the DSS in an exit survey. Our results show that participants tend to score better with the DSS, that the score increase is due to users following the DSS advice, and related to the difficulty of the game and the accuracy of the DSS. We observe that this setting elicits mostly rational behavior from participants, who place a moderate amount of trust in the DSS and show neither algorithmic aversion (under-reliance) nor automation bias (over-reliance).However, their stated willingness to accept the DSS in the exit survey seems less sensitive to the accuracy of the DSS than their behavior, suggesting that users are only partially aware of the (lack of) accuracy of the DSS.

Information technology support for collaborative decision making in auditing: An experimental investigation

An Analysis of the Group Dynamics Surrounding Internal Control Assessment in Information Systems Audit and Assurance Domains

THE EFFECTS OF GROUP DECISION SUPPORT SYSTEM (GDSS)AND GROUP DISCUSSION ON GROUP DECISION MAKING

Applications and Implementation DECISION SUPPORT SYSTEMS: AN EMPIRICAL ASSESSMENT FOR DECISION MAKING

RoundTable: Investigating Group Decision-Making Mechanism in Multi-Agent Collaboration

The influence of the shared-display configuration on group decision making

A web‐based group decision support system for multicriteria problems

Group Decision Making in Information Systems Security Assessment Using Dual Hesitant Fuzzy Set.

Managing non-cooperative behaviors in large-scale group decision making based on trust relationships and confidence levels of decision makers

Shared Experiences of Technology and Trust: an Experimental Study of Physiological Compliance Between Active and Passive Users in Technology-Mediated Collaborative Encounters

Towards a Reliable Framework of Uncertainty-Based Group Decision Support System

Human Response to an AI-Based Decision Support System: A User Study on the Effects of Accuracy and Bias

Group Decisions based on Confidence Weighted Majority Voting

A Framework for Adopting Collaboration 2.0 Tools for Virtual Group Decision Making

Group Decision-Making Method with Trust-Based Weight and Reliability Parameters

Anytime collaborative brain-computer interfaces for enhancing perceptual group decision-making

Technology-Supported Self-Triage Decision Making: A Mixed-Methods Study

Dynamics of Trust in Automation and Interactive Decision Making During Driving Simulation Tasks

A Reliable System Platform for Group Decision Support under Uncertain Environments

Decision Model for Complex Group Argumentation.

Experiential Benefits of Interactive Conflict Negotiation Practices in Computer-Supported Shift Planning