Abstract:Explainable artificial intelligence (XAI) has been increasingly investigated to enhance the transparency of black-box artificial intelligence models, promoting better user understanding and trust. Developing an XAI that is faithful to models and plausible to users is both a necessity and a challenge. This work examines whether embedding human attention knowledge into saliency-based XAI methods for computer vision models could enhance their plausibility and faithfulness. Two novel XAI methods for object detection models, namely FullGrad-CAM and FullGrad-CAM++, were first developed to generate object-specific explanations by extending the current gradient-based XAI methods for image classification models. Using human attention as the objective plausibility measure, these methods achieve higher explanation plausibility. Interestingly, all current XAI methods when applied to object detection models generally produce saliency maps that are less faithful to the model than human attention maps from the same object detection task. Accordingly, human attention-guided XAI (HAG-XAI) was proposed to learn from human attention how to best combine explanatory information from the models to enhance explanation plausibility by using trainable activation functions and smoothing kernels to maximize the similarity between XAI saliency map and human attention map. The proposed XAI methods were evaluated on widely used BDD-100K, MS-COCO, and ImageNet datasets and compared with typical gradient-based and perturbation-based XAI methods. Results suggest that HAG-XAI enhanced explanation plausibility and user trust at the expense of faithfulness for image classification models, and it enhanced plausibility, faithfulness, and user trust simultaneously and outperformed existing state-of-the-art XAI methods for object detection models.

How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency maps

Unraveling the Dilemma of AI Errors: Exploring the Effectiveness of Human and Machine Explanations for Large Language Models

Explanation strategies in humans versus current explainable artificial intelligence: Insights from image classification

Interpretability is in the eye of the beholder: Human versus artificial classification of image segments generated by humans versus XAI

The benefits and costs of explainable artificial intelligence in visual quality control: Evidence from fault detection performance and eye movements

A Meta-Analysis of the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making

Explanation Strategies for Image Classification in Humans vs. Current Explainable AI

Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Study

Evaluating the Impact of Human Explanation Strategies on Human-AI Visual Decision-Making

Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of xAI on Human-Agent Interaction

Human Learning about AI Performance

The Impact of Imperfect XAI on Human-AI Decision-Making

The Limits of Perception: Analyzing Inconsistencies in Saliency Maps in XAI

Explainable AI improves task performance in human-AI collaboration

The role of saliency maps in enhancing ophthalmologists' trust in artificial intelligence models

A Psychological Theory of Explainability

Human attention guided explainable artificial intelligence for computer vision models

Explainable GeoAI: can saliency maps help interpret artificial intelligence's learning process? An empirical study on natural feature detection

Evaluating the Influences of Explanation Style on Human-AI Reliance

Explain To Decide: A Human-Centric Review on the Role of Explainable Artificial Intelligence in AI-assisted Decision Making