Maintenance of transmitter release from neuromuscular junctions with different patterns of usage ‘in vivo’

G. Bewick

DOI: https://doi.org/10.1023/B:NEUR.0000020605.20628.A3

2003-06-01

Journal of Neurocytology

Abstract:

What problem does this paper attempt to address?

Impacts of AI-Generated Confidence and Explanations on Task Performance and Trust in Human-autonomy Teaming

Shihong Ling,Yutong Zhang,Na Du

DOI: https://doi.org/10.1177/21695067231195002

2023-10-24

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

Abstract:In human-autonomy teaming (HAT), human operators and intelligent agents cooperate and coordinate together to achieve shared goals. This study aimed to enhance autonomy transparency by proposing the automatic generation of confidence and explanations (bounding boxes, and bounding boxes and keypoints) and investigating their impacts in HAT. A total of 36 participants engaged in a simulated surveillance task, during which they were assisted by the intelligent agents we designed using Keypoint Faster R-CNN. As a result, we found that visual explanations using bounding boxes and keypoints improved detection task performance only when confidence was not visualized. Moreover, participants had higher trust in and preference for autonomy when visual explanations were provided but whether confidence was visualized did not influence their trust and preference. These findings have implications for the design of autonomy and can facilitate human-machine interactions in HAT.
A Tale of Two Explanations: Enhancing Human Trust by Explaining Robot Behavior.

Mark Edmonds,Feng Gao,Hangxin Liu,Xu Xie,Siyuan Qi,Brandon Rothrock,Yixin Zhu,Ying Nian Wu,Hongjing Lu,Song-Chun Zhu

DOI: https://doi.org/10.1126/scirobotics.aay4663

IF: 25

2019-01-01

Science Robotics

Abstract:The ability to provide comprehensive explanations of chosen actions is a hallmark of intelligence. Lack of this ability impedes the general acceptance of AI and robot systems in critical tasks. This paper examines what forms of explanations best foster human trust in machines and proposes a framework in which explanations are generated from both functional and mechanistic perspectives. The robot system learns from human demonstrations to open medicine bottles using (i) an embodied haptic prediction model to extract knowledge from sensory feedback, (ii) a stochastic grammar model induced to capture the compositional structure of a multistep task, and (iii) an improved Earley parsing algorithm to jointly leverage both the haptic and grammar models. The robot system not only shows the ability to learn from human demonstrators but also succeeds in opening new, unseen bottles. Using different forms of explanations generated by the robot system, we conducted a psychological experiment to examine what forms of explanations best foster human trust in the robot. We found that comprehensive and real-time visualizations of the robot’s internal decisions were more effective in promoting human trust than explanations based on summary text descriptions. In addition, forms of explanation that are best suited to foster trust do not necessarily correspond to the model components contributing to the best task performance. This divergence shows a need for the robotics community to integrate model components to enhance both task execution and human trust in machines.
Exploring the effects of human-centered AI explanations on trust and reliance

Nicolas Scharowski,Sebastian A. C. Perrig,Melanie Svab,Klaus Opwis,Florian Brühlmann

DOI: https://doi.org/10.3389/fcomp.2023.1151150

2023-07-17

Frontiers in Computer Science

Abstract:Transparency is widely regarded as crucial for the responsible real-world deployment of artificial intelligence (AI) and is considered an essential prerequisite to establishing trust in AI. There are several approaches to enabling transparency, with one promising attempt being human-centered explanations. However, there is little research into the effectiveness of human-centered explanations on end-users' trust. What complicates the comparison of existing empirical work is that trust is measured in different ways. Some researchers measure subjective trust using questionnaires, while others measure objective trust-related behavior such as reliance. To bridge these gaps, we investigated the effects of two promising human-centered post-hoc explanations, feature importance and counterfactuals , on trust and reliance. We compared these two explanations with a control condition in a decision-making experiment ( N = 380). Results showed that human-centered explanations can significantly increase reliance but the type of decision-making (increasing a price vs. decreasing a price) had an even greater influence. This challenges the presumed importance of transparency over other factors in human decision-making involving AI, such as potential heuristics and biases. We conclude that trust does not necessarily equate to reliance and emphasize the importance of appropriate, validated, and agreed-upon metrics to design and evaluate human-centered AI.
Understanding the influence of AI autonomy on AI explainability levels in human-AI teams using a mixed methods approach

Allyson I. Hauptman,Beau G. Schelble,Wen Duan,Christopher Flathmann,Nathan J. McNeese

DOI: https://doi.org/10.1007/s10111-024-00765-7

2024-05-19

Cognition Technology & Work

Abstract:An obstacle to effective teaming between humans and AI is the agent's "black box" design. AI explanations have proven benefits, but few studies have explored the effects that explanations can have in a teaming environment with AI agents operating at heightened levels of autonomy. We conducted two complementary studies, an experiment and participatory design sessions, investigating the effect that varying levels of AI explainability and AI autonomy have on the participants' perceived trust and competence of an AI teammate to address this research gap. The results of the experiment were counter-intuitive, where the participants actually perceived the lower explainability agent as both more trustworthy and more competent. The participatory design sessions further revealed how a team's need to know influences when and what teammates need explained from AI teammates. Based on these findings, several design recommendations were developed for the HCI community to guide how AI teammates should share decision information with their human counterparts considering the careful balance between trust and competence in human-AI teams.

engineering, industrial,ergonomics
How transparency modulates trust in artificial intelligence

John Zerilli,Umang Bhatt,Adrian Weller

DOI: https://doi.org/10.1016/j.patter.2022.100455

2022-02-24

Abstract:The study of human-machine systems is central to a variety of behavioral and engineering disciplines, including management science, human factors, robotics, and human-computer interaction. Recent advances in artificial intelligence (AI) and machine learning have brought the study of human-AI teams into sharper focus. An important set of questions for those designing human-AI interfaces concerns trust, transparency, and error tolerance. Here, we review the emerging literature on this important topic, identify open questions, and discuss some of the pitfalls of human-AI team research. We present opposition (extreme algorithm aversion or distrust) and loafing (extreme automation complacency or bias) as lying at opposite ends of a spectrum, with algorithmic vigilance representing an ideal mid-point. We suggest that, while transparency may be crucial for facilitating appropriate levels of trust in AI and thus for counteracting aversive behaviors and promoting vigilance, transparency should not be conceived solely in terms of the explainability of an algorithm. Dynamic task allocation, as well as the communication of confidence and performance metrics-among other strategies-may ultimately prove more useful to users than explanations from algorithms and significantly more effective in promoting vigilance. We further suggest that, while both aversive and appreciative attitudes are detrimental to optimal human-AI team performance, strategies to curb aversion are likely to be more important in the longer term than those attempting to mitigate appreciation. Our wider aim is to channel disparate efforts in human-AI team research into a common framework and to draw attention to the ecological validity of results in this field.
A New Method of Computing Fundamental Units in Algebraic Number Fields

U. Fincke,M. Pohst

DOI: https://doi.org/10.1007/3-540-15984-3_315

1985-04-01

Abstract:
Incorporating Explanations into Human-Machine Interfaces for Trust and Situation Awareness in Autonomous Vehicles

Shahin Atakishiyev,Mohammad Salameh,Randy Goebel

2024-04-11

Abstract:Autonomous vehicles often make complex decisions via machine learning-based predictive models applied to collected sensor data. While this combination of methods provides a foundation for real-time actions, self-driving behavior primarily remains opaque to end users. In this sense, explainability of real-time decisions is a crucial and natural requirement for building trust in autonomous vehicles. Moreover, as autonomous vehicles still cause serious traffic accidents for various reasons, timely conveyance of upcoming hazards to road users can help improve scene understanding and prevent potential risks. Hence, there is also a need to supply autonomous vehicles with user-friendly interfaces for effective human-machine teaming. Motivated by this problem, we study the role of explainable AI and human-machine interface jointly in building trust in vehicle autonomy. We first present a broad context of the explanatory human-machine systems with the "3W1H" (what, whom, when, how) approach. Based on these findings, we present a situation awareness framework for calibrating users' trust in self-driving behavior. Finally, we perform an experiment on our framework, conduct a user study on it, and validate the empirical findings with hypothesis testing.

Robotics,Artificial Intelligence
Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces

Alexander Kunze,Stephen J. Summerskill,Russell Marshall,Ashleigh J. Filtness

DOI: https://doi.org/10.1080/00140139.2018.1547842

IF: 2.5612

2019-02-05

Ergonomics

Abstract:Operators of highly automated driving systems may exhibit behaviour characteristic for overtrust issues due to an insufficient awareness of automation fallibility. Consequently, situation awareness in critical situations is reduced and safe driving performance following emergency takeovers is impeded. A driving simulator study was used to assess the impact of dynamically communicating system uncertainties on monitoring, trust, workload, takeovers, and physiological responses. The uncertainty information was conveyed visually using a stylised heart beat combined with a numerical display and users were engaged in a visual search task. Multilevel analysis results suggest that uncertainty communication helps operators calibrate their trust and gain situation awareness prior to critical situations, resulting in safer takeovers. In addition, eye tracking data indicate that operators can adjust their gaze behaviour in correspondence with the level of uncertainty. However, conveying uncertainties using a visual display significantly increases operator workload and impedes users in the execution of non-driving related tasks. Practitioner Summary: This article illustrates how the communication of system uncertainty information helps operators calibrate their trust in automation and, consequently, gain situation awareness. Multilevel analysis results of a driving simulator study affirm the benefits for trust calibration and highlight that operators adjust their behaviour according to multiple uncertainty levels.

engineering, industrial,ergonomics,psychology, applied
On the Importance of User Backgrounds and Impressions: Lessons Learned from Interactive AI Applications

Mahsan Nourani,Chiradeep Roy,Jeremy E. Block,Donald R. Honeycutt,Tahrima Rahman,Eric D. Ragan,Vibhav Gogate

DOI: https://doi.org/10.1145/3531066

IF: 3.4

2022-04-29

ACM Transactions on Interactive Intelligent Systems

Abstract:While EXplainable Artificial Intelligence (XAI) approaches aim to improve human-AI collaborative decision-making by improving model transparency and mental model formations, experiential factors associated with human users can cause challenges in ways system designers do not anticipate. In this paper, we first showcase a user study on how anchoring bias can potentially affect mental model formations when users initially interact with an intelligent system and the role of explanations in addressing this bias. Using a video activity recognition tool in cooking domain, we asked participants to verify whether a set of kitchen policies are being followed, with each policy focusing on a weakness or a strength. We controlled the order of the policies and the presence of explanations to test our hypotheses. Our main finding shows that those who observed system strengths early-on were more prone to automation bias and made significantly more errors due to positive first impressions of the system, while they built a more accurate mental model of the system competencies. On the other hand, those who encountered weaknesses earlier made significantly fewer errors since they tended to rely more on themselves, while they also underestimated model competencies due to having a more negative first impression of the model. Motivated by these findings and similar existing work, we formalize and present a conceptual model of user’s past experiences that examine the relations between user’s backgrounds, experiences, and human factors in XAI systems based on usage time. Our work presents strong findings and implications, aiming to raise the awareness of AI designers towards biases associated with user impressions and backgrounds.

computer science, artificial intelligence
Effect of AI Explanations on Human Perceptions of Patient-Facing AI-Powered Healthcare Systems

Zhan Zhang,Yegin Genc,Dakuo Wang,Mehmet Eren Ahsen,Xiangmin Fan

DOI: https://doi.org/10.1007/s10916-021-01743-6

IF: 4.92

2021-05-04

Journal of Medical Systems

Abstract:Ongoing research efforts have been examining how to utilize artificial intelligence technology to help healthcare consumers make sense of their clinical data, such as diagnostic radiology reports. How to promote the acceptance of such novel technology is a heated research topic. Recent studies highlight the importance of providing local explanations about AI prediction and model performance to help users determine whether to trust AI's predictions. Despite some efforts, limited empirical research has been conducted to quantitatively measure how AI explanations impact healthcare consumers' perceptions of using patient-facing, AI-powered healthcare systems. The aim of this study is to evaluate the effects of different AI explanations on people's perceptions of AI-powered healthcare system. In this work, we designed and deployed a large-scale experiment (N = 3,423) on Amazon Mechanical Turk (MTurk) to evaluate the effects of AI explanations on people's perceptions in the context of comprehending radiology reports. We created four groups based on two factors—the extent of explanations for the prediction (High vs. Low Transparency) and the model performance (Good vs. Weak AI Model)—and randomly assigned participants to one of the four conditions. Participants were instructed to classify a radiology report as describing a normal or abnormal finding, followed by completing a post-study survey to indicate their perceptions of the AI tool. We found that revealing model performance information can promote people's trust and perceived usefulness of system outputs, while providing local explanations for the rationale of a prediction can promote understandability but not necessarily trust. We also found that when model performance is low, the more information the AI system discloses, the less people would trust the system. Lastly, whether human agrees with AI predictions or not and whether the AI prediction is correct or not could also influence the effect of AI explanations. We conclude this paper by discussing implications for designing AI systems for healthcare consumers to interpret diagnostic report.

health care sciences & services,medical informatics
Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making

Yunfeng Zhang,Q. Vera Liao,Rachel K. E. Bellamy

DOI: https://doi.org/10.1145/3351095.3372852

2020-01-22

Abstract:Today, AI is being increasingly used to help human experts make decisions in high-stakes scenarios. In these scenarios, full automation is often undesirable, not only due to the significance of the outcome, but also because human experts can draw on their domain knowledge complementary to the model's to ensure task success. We refer to these scenarios as AI-assisted decision making, where the individual strengths of the human and the AI come together to optimize the joint decision outcome. A key to their success is to appropriately calibrate human trust in the AI on a case-by-case basis; knowing when to trust or distrust the AI allows the human expert to appropriately apply their knowledge, improving decision outcomes in cases where the model is likely to perform poorly. This research conducts a case study of AI-assisted decision making in which humans and AI have comparable performance alone, and explores whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. Specifically, we study the effect of showing confidence score and local explanation for a particular prediction. Through two human experiments, we show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making, which may also depend on whether the human can bring in enough unique knowledge to complement the AI's errors. We also highlight the problems in using local explanation for AI-assisted decision making scenarios and invite the research community to explore new approaches to explainability for calibrating human trust in AI.
Not All Information Is Equal: Effects of Disclosing Different Types of Likelihood Information on Trust, Compliance and Reliance, and Task Performance in Human-Automation Teaming

Na Du,Kevin Y. Huang,X. Jessie Yang

DOI: https://doi.org/10.1177/0018720819862916

2019-07-26

Abstract:Objective The study examines the effects of disclosing different types of likelihood information on human operators’ trust in automation, their compliance and reliance behaviors, and the human-automation team performance. Background To facilitate appropriate trust in and dependence on automation, explicitly conveying the likelihood of automation success has been proposed as one solution. Empirical studies have been conducted to investigate the potential benefits of disclosing likelihood information in the form of automation reliability, (un)certainty, and confidence. Yet, results from these studies are rather mixed. Method We conducted a human-in-the-loop experiment with 60 participants using a simulated surveillance task. Each participant performed a compensatory tracking task and a threat detection task with the help of an imperfect automated threat detector. Three types of likelihood information were presented: overall likelihood information, predictive values, and hit and correct rejection rates. Participants’ trust in automation, compliance and reliance behaviors, and task performance were measured. Results Human operators informed of the predictive values or the overall likelihood value, rather than the hit and correct rejection rates, relied on the decision aid more appropriately and obtained higher task scores. Conclusion Not all likelihood information is equal in aiding human-automation team performance. Directly presenting the hit and correct rejection rates of an automated decision aid should be avoided. Application The findings can be applied to the design of automated decision aids.
Look Who's Talking Now: Implications of AV's Explanations on Driver's Trust, AV Preference, Anxiety and Mental Workload

Na Du,Jacob Haspiel,Qiaoning Zhang,Dawn Tilbury,Anuj K. Pradhan,X. Jessie Yang,Lionel P. Robert Jr

DOI: https://doi.org/10.48550/arXiv.1905.08878

2019-05-22

Abstract:Explanations given by automation are often used to promote automation adoption. However, it remains unclear whether explanations promote acceptance of automated vehicles (AVs). In this study, we conducted a within-subject experiment in a driving simulator with 32 participants, using four different conditions. The four conditions included: (1) no explanation, (2) explanation given before or (3) after the AV acted and (4) the option for the driver to approve or disapprove the AV's action after hearing the explanation. We examined four AV outcomes: trust, preference for AV, anxiety and mental workload. Results suggest that explanations provided before an AV acted were associated with higher trust in and preference for the AV, but there was no difference in anxiety and workload. These results have important implications for the adoption of AVs.

Human-Computer Interaction,Computers and Society,Robotics
Reliable and transparent in-vehicle agents lead to higher behavioral trust in conditionally automated driving systems

Skye Taylor,Manhua Wang,Myounghoon Jeon

DOI: https://doi.org/10.3389/fpsyg.2023.1121622

IF: 3.8

2023-05-18

Frontiers in Psychology

Abstract:Trust is critical for human-automation collaboration, especially under safety-critical tasks such as driving. Providing explainable information on how the automation system reaches decisions and predictions can improve system transparency, which is believed to further facilitate driver trust and user evaluation of the automated vehicles. However, what the optimal level of transparency is and how the system communicates it to calibrate drivers’ trust and improve their driving performance remain uncertain. Such uncertainty becomes even more unpredictable given that the system reliability remains dynamic due to current technological limitations. To address this issue in conditionally automated vehicles, a total of 30 participants were recruited in a driving simulator study and assigned to either a low or a high system reliability condition. They experienced two driving scenarios accompanied by two types of in-vehicle agents delivering information with different transparency types: “what”-then-wait (on-demand) and “what + why” (proactive). The on-demand agent provided some information about the upcoming event and delivered more information if prompted by the driver, whereas the proactive agent provided all information at once. Results indicated that the on-demand agent was more habitable, or naturalistic, to drivers and was perceived with faster system response speed compared to the proactive agent. Drivers under the high-reliability condition complied with the takeover request (TOR) more (if the agent was on-demand) and had shorter takeover times (in both agent conditions) compared to those under the low-reliability condition. These findings inspire how the automation system can deliver information to improve system transparency while adapting to system reliability and user evaluation, which further contributes to driver trust calibration and performance correction in future automated vehicles.

psychology, multidisciplinary
A Transparency Paradox? Investigating the Impact of Explanation Specificity and Autonomous Vehicle Perceptual Inaccuracies on Passengers

Daniel Omeiza,Raunak Bhattacharyya,Marina Jirotka,Nick Hawes,Lars Kunze

2024-08-16

Abstract:Transparency in automated systems could be afforded through the provision of intelligible explanations. While transparency is desirable, might it lead to catastrophic outcomes (such as anxiety), that could outweigh its benefits? It's quite unclear how the specificity of explanations (level of transparency) influences recipients, especially in autonomous driving (AD). In this work, we examined the effects of transparency mediated through varying levels of explanation specificity in AD. We first extended a data-driven explainer model by adding a rule-based option for explanation generation in AD, and then conducted a within-subject lab study with 39 participants in an immersive driving simulator to study the effect of the resulting explanations. Specifically, our investigation focused on: (1) how different types of explanations (specific vs. abstract) affect passengers' perceived safety, anxiety, and willingness to take control of the vehicle when the vehicle perception system makes erroneous predictions; and (2) the relationship between passengers' behavioural cues and their feelings during the autonomous drives. Our findings showed that passengers felt safer with specific explanations when the vehicle's perception system had minimal errors, while abstract explanations that hid perception errors led to lower feelings of safety. Anxiety levels increased when specific explanations revealed perception system errors (high transparency). We found no significant link between passengers' visual patterns and their anxiety levels. Our study suggests that passengers prefer clear and specific explanations (high transparency) when they originate from autonomous vehicles (AVs) with optimal perceptual accuracy.

Human-Computer Interaction,Artificial Intelligence,Computers and Society,Machine Learning,Robotics
Don't be Fooled: The Misinformation Effect of Explanations in Human-AI Collaboration

Philipp Spitzer,Joshua Holstein,Katelyn Morrison,Kenneth Holstein,Gerhard Satzger,Niklas Kühl

2024-09-19

Abstract:Across various applications, humans increasingly use black-box artificial intelligence (AI) systems without insight into these systems' reasoning. To counter this opacity, explainable AI (XAI) methods promise enhanced transparency and interpretability. While recent studies have explored how XAI affects human-AI collaboration, few have examined the potential pitfalls caused by incorrect explanations. The implications for humans can be far-reaching but have not been explored extensively. To investigate this, we ran a study (n=160) on AI-assisted decision-making in which humans were supported by XAI. Our findings reveal a misinformation effect when incorrect explanations accompany correct AI advice with implications post-collaboration. This effect causes humans to infer flawed reasoning strategies, hindering task execution and demonstrating impaired procedural knowledge. Additionally, incorrect explanations compromise human-AI team-performance during collaboration. With our work, we contribute to HCI by providing empirical evidence for the negative consequences of incorrect explanations on humans post-collaboration and outlining guidelines for designers of AI.

Human-Computer Interaction,Artificial Intelligence
Explain yourself! Effects of Explanations in Human-Robot Interaction

Jakob Ambsdorf,Alina Munir,Yiyao Wei,Klaas Degkwitz,Harm Matthias Harms,Susanne Stannek,Kyra Ahrens,Dennis Becker,Erik Strahl,Tom Weber,Stefan Wermter

DOI: https://doi.org/10.48550/arXiv.2204.04501

2022-06-14

Abstract:Recent developments in explainable artificial intelligence promise the potential to transform human-robot interaction: Explanations of robot decisions could affect user perceptions, justify their reliability, and increase trust. However, the effects on human perceptions of robots that explain their decisions have not been studied thoroughly. To analyze the effect of explainable robots, we conduct a study in which two simulated robots play a competitive board game. While one robot explains its moves, the other robot only announces them. Providing explanations for its actions was not sufficient to change the perceived competence, intelligence, likeability or safety ratings of the robot. However, the results show that the robot that explains its moves is perceived as more lively and human-like. This study demonstrates the need for and potential of explainable human-robot interaction and the wider assessment of its effects as a novel research direction.

Robotics,Machine Learning
Critical Roles of Explainability in Shaping Perception, Trust, and Acceptance of Autonomous Vehicles

Tingru Zhang,Weitao Li,Weixing Huang,Liang Ma

DOI: https://doi.org/10.1016/j.ergon.2024.103568

IF: 2.884

2024-01-01

International Journal of Industrial Ergonomics

Abstract:Despite the advancements in autonomous vehicles (AVs) and their potential benefits, widespread acceptance of AVs remains low due to the significant barrier of trust. While prior research has explored various factors influencing trust towards AVs, the role of explainability—AVs' ability to describe the rationale behind their outputs in human-understandable terms—has been largely overlooked. This study aimed to investigate how the perceived explainability of AVs impacts driver perception, trust, and the acceptance of AVs. For this end, an enhanced AV acceptance model that incorporates novel features such as perceived explainability and perceived intelligence was proposed. In order to validate the proposed model, a survey was conducted in which participants were exposed to different AV introductions (basic introduction, video introduction, or introduction with how + why explanations). The responses of 399 participants were analyzed using structural equation modeling. The results showed that perceived explainability had the most profound impact on trust, exerting both direct and indirect effects. AVs perceived as more explainable were also considered easier to use, more useful, safer, and more intelligent, which in turn fostered trust and acceptance. Additionally, the impact of perceived intelligence on trust was significant, indicating that drivers view AVs as intelligent agents rather than mere passive tools. While traditional factors such as perceived ease of use and perceived usefulness remained significant predictors of trust, their effects were smaller than perceived explainability and perceived intelligence. These findings underscore the importance of considering the role of explainability in AV design to address the trust-related challenges associated with AV adoption.
Confident Teacher, Confident Student? A Novel User Study Design for Investigating the Didactic Potential of Explanations and their Impact on Uncertainty

Teodor Chiaburu,Frank Haußer,Felix Bießmann

2024-09-10

Abstract:Evaluating the quality of explanations in Explainable Artificial Intelligence (XAI) is to this day a challenging problem, with ongoing debate in the research community. While some advocate for establishing standardized offline metrics, others emphasize the importance of human-in-the-loop (HIL) evaluation. Here we propose an experimental design to evaluate the potential of XAI in human-AI collaborative settings as well as the potential of XAI for didactics. In a user study with 1200 participants we investigate the impact of explanations on human performance on a challenging visual task - annotation of biological species in complex taxonomies. Our results demonstrate the potential of XAI in complex visual annotation tasks: users become more accurate in their annotations and demonstrate less uncertainty with AI assistance. The increase in accuracy was, however, not significantly different when users were shown the mere prediction of the model compared to when also providing an explanation. We also find negative effects of explanations: users tend to replicate the model's predictions more often when shown explanations, even when those predictions are wrong. When evaluating the didactic effects of explanations in collaborative human-AI settings, we find that users' annotations are not significantly better after performing annotation with AI assistance. This suggests that explanations in visual human-AI collaboration do not appear to induce lasting learning effects. All code and experimental data can be found in our GitHub repository: <a class="link-external link-https" href="https://github.com/TeodorChiaburu/beexplainable" rel="external noopener nofollow">this https URL</a>.

Human-Computer Interaction,Artificial Intelligence,Computers and Society
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations

Valerie Chen,Q. Vera Liao,Jennifer Wortman Vaughan,Gagan Bansal

DOI: https://doi.org/10.48550/arXiv.2301.07255

2023-06-14

Abstract:AI explanations are often mentioned as a way to improve human-AI decision-making, but empirical studies have not found consistent evidence of explanations' effectiveness and, on the contrary, suggest that they can increase overreliance when the AI system is wrong. While many factors may affect reliance on AI support, one important factor is how decision-makers reconcile their own intuition -- beliefs or heuristics, based on prior knowledge, experience, or pattern recognition, used to make judgments -- with the information provided by the AI system to determine when to override AI predictions. We conduct a think-aloud, mixed-methods study with two explanation types (feature- and example-based) for two prediction tasks to explore how decision-makers' intuition affects their use of AI predictions and explanations, and ultimately their choice of when to rely on AI. Our results identify three types of intuition involved in reasoning about AI predictions and explanations: intuition about the task outcome, features, and AI limitations. Building on these, we summarize three observed pathways for decision-makers to apply their own intuition and override AI predictions. We use these pathways to explain why (1) the feature-based explanations we used did not improve participants' decision outcomes and increased their overreliance on AI, and (2) the example-based explanations we used improved decision-makers' performance over feature-based explanations and helped achieve complementary human-AI performance. Overall, our work identifies directions for further development of AI decision-support systems and explanation methods that help decision-makers effectively apply their intuition to achieve appropriate reliance on AI.

Human-Computer Interaction,Artificial Intelligence

Maintenance of transmitter release from neuromuscular junctions with different patterns of usage ‘in vivo’

Impacts of AI-Generated Confidence and Explanations on Task Performance and Trust in Human-autonomy Teaming

A Tale of Two Explanations: Enhancing Human Trust by Explaining Robot Behavior.

Exploring the effects of human-centered AI explanations on trust and reliance

Understanding the influence of AI autonomy on AI explainability levels in human-AI teams using a mixed methods approach

How transparency modulates trust in artificial intelligence

A New Method of Computing Fundamental Units in Algebraic Number Fields

Incorporating Explanations into Human-Machine Interfaces for Trust and Situation Awareness in Autonomous Vehicles

Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces

On the Importance of User Backgrounds and Impressions: Lessons Learned from Interactive AI Applications

Effect of AI Explanations on Human Perceptions of Patient-Facing AI-Powered Healthcare Systems

Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making

Not All Information Is Equal: Effects of Disclosing Different Types of Likelihood Information on Trust, Compliance and Reliance, and Task Performance in Human-Automation Teaming

Look Who's Talking Now: Implications of AV's Explanations on Driver's Trust, AV Preference, Anxiety and Mental Workload

Reliable and transparent in-vehicle agents lead to higher behavioral trust in conditionally automated driving systems

A Transparency Paradox? Investigating the Impact of Explanation Specificity and Autonomous Vehicle Perceptual Inaccuracies on Passengers

Don't be Fooled: The Misinformation Effect of Explanations in Human-AI Collaboration

Explain yourself! Effects of Explanations in Human-Robot Interaction

Critical Roles of Explainability in Shaping Perception, Trust, and Acceptance of Autonomous Vehicles

Confident Teacher, Confident Student? A Novel User Study Design for Investigating the Didactic Potential of Explanations and their Impact on Uncertainty

Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations