Abstract:Ongoing research efforts have been examining how to utilize artificial intelligence technology to help healthcare consumers make sense of their clinical data, such as diagnostic radiology reports. How to promote the acceptance of such novel technology is a heated research topic. Recent studies highlight the importance of providing local explanations about AI prediction and model performance to help users determine whether to trust AI's predictions. Despite some efforts, limited empirical research has been conducted to quantitatively measure how AI explanations impact healthcare consumers' perceptions of using patient-facing, AI-powered healthcare systems. The aim of this study is to evaluate the effects of different AI explanations on people's perceptions of AI-powered healthcare system. In this work, we designed and deployed a large-scale experiment (N = 3,423) on Amazon Mechanical Turk (MTurk) to evaluate the effects of AI explanations on people's perceptions in the context of comprehending radiology reports. We created four groups based on two factors—the extent of explanations for the prediction (High vs. Low Transparency) and the model performance (Good vs. Weak AI Model)—and randomly assigned participants to one of the four conditions. Participants were instructed to classify a radiology report as describing a normal or abnormal finding, followed by completing a post-study survey to indicate their perceptions of the AI tool. We found that revealing model performance information can promote people's trust and perceived usefulness of system outputs, while providing local explanations for the rationale of a prediction can promote understandability but not necessarily trust. We also found that when model performance is low, the more information the AI system discloses, the less people would trust the system. Lastly, whether human agrees with AI predictions or not and whether the AI prediction is correct or not could also influence the effect of AI explanations. We conclude this paper by discussing implications for designing AI systems for healthcare consumers to interpret diagnostic report.

CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysis

Outlining the Design Space of Explainable Intelligent Systems for Medical Diagnosis

Evaluating Explainable Artificial Intelligence (XAI) techniques in chest radiology imaging through a human-centered Lens

Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond

Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks

Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions

Care to Explain? AI Explanation Types Differentially Impact Chest Radiograph Diagnostic Performance and Physician Trust in AI

Explainable Artificial Intelligence for Medical Applications: A Review

A Survey on Medical Explainable AI (XAI): Recent Progress, Explainability Approach, Human Interaction and Scoring System

Co-design of human-centered, explainable AI for clinical decision support

Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosis

Dissection of medical AI reasoning processes via physician and generative-AI collaboration

A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI

Transparent and Clinically Interpretable AI for Lung Cancer Detection in Chest X-Rays

Explainable AI applications in the Medical Domain: a systematic review

Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting

Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging

Effect of AI Explanations on Human Perceptions of Patient-Facing AI-Powered Healthcare Systems

Unveiling the black box: A systematic review of Explainable Artificial Intelligence in medical image analysis

What Are the Users’ Needs? Design of a User-Centered Explainable Artificial Intelligence Diagnostic System