Abstract:While EXplainable Artificial Intelligence (XAI) approaches aim to improve human-AI collaborative decision-making by improving model transparency and mental model formations, experiential factors associated with human users can cause challenges in ways system designers do not anticipate. In this paper, we first showcase a user study on how anchoring bias can potentially affect mental model formations when users initially interact with an intelligent system and the role of explanations in addressing this bias. Using a video activity recognition tool in cooking domain, we asked participants to verify whether a set of kitchen policies are being followed, with each policy focusing on a weakness or a strength. We controlled the order of the policies and the presence of explanations to test our hypotheses. Our main finding shows that those who observed system strengths early-on were more prone to automation bias and made significantly more errors due to positive first impressions of the system, while they built a more accurate mental model of the system competencies. On the other hand, those who encountered weaknesses earlier made significantly fewer errors since they tended to rely more on themselves, while they also underestimated model competencies due to having a more negative first impression of the model. Motivated by these findings and similar existing work, we formalize and present a conceptual model of user’s past experiences that examine the relations between user’s backgrounds, experiences, and human factors in XAI systems based on usage time. Our work presents strong findings and implications, aiming to raise the awareness of AI designers towards biases associated with user impressions and backgrounds.

Gamifying XAI: Enhancing AI Explainability for Non-technical Users through LLM-Powered Narrative Gamifications

Explainable AI for Cheating Detection and Churn Prediction in Online Games

Getting Playful with Explainable AI: Games with a Purpose to Improve Human Understanding of AI

Explainability via Interactivity? Supporting Nonexperts' Sensemaking of Pretrained CNN by Interacting with Their Daily Surroundings

Tell Me a Story! Narrative-Driven XAI with Large Language Models

On the Importance of User Backgrounds and Impressions: Lessons Learned from Interactive AI Applications

Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era

Explainable Artificial Intelligence (XAI) for Increasing User Trust in Deep Reinforcement Learning Driven Autonomous Systems

Explainable Interfaces for Rapid Gaze-Based Interactions in Mixed Reality

Explainability in JupyterLab and Beyond: Interactive XAI Systems for Integrated and Collaborative Workflows

Explainable AI for tailored electricity consumption feedback -- an experimental evaluation of visualizations

Bridging AI Developers and End Users: an End-User-Centred Explainable AI Taxonomy and Visual Vocabularies

Toward enriched Cognitive Learning with XAI

"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction

Improving User Mental Models of XAI Systems with an Inclusive Design Approach

Interpretable Narrative Explanation for ML Predictors with LP: A Case Study for XAI

User-centric AIGC products: Explainable Artificial Intelligence and AIGC products

Transcending XAI Algorithm Boundaries through End-User-Inspired Design

User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical study

Deliberative XAI: How Explanations Impact Understanding and Decision-Making of AI Novices in Collective and Individual Settings

Effective Representation to Capture Collaboration Behaviors between Explainer and User