Abstract:Background: Artificial Intelligence has the potential to revolutionize healthcare, and it is increasingly being deployed to support and assist medical diagnosis. One potential application of AI is as the first point of contact for patients, replacing initial diagnoses prior to sending a patient to a specialist, allowing health care professionals to focus on more challenging and critical aspects of treatment. But for AI systems to succeed in this role, it will not be enough for them to merely provide accurate diagnoses and predictions. In addition, it will need to provide explanations (both to physicians and patients) about why the diagnoses are made. Without this, accurate and correct diagnoses and treatments might otherwise be ignored or rejected. Method: It is important to evaluate the effectiveness of these explanations and understand the relative effectiveness of different kinds of explanations. In this paper, we examine this problem across two simulation experiments. For the first experiment, we tested a re-diagnosis scenario to understand the effect of local and global explanations. In a second simulation experiment, we implemented different forms of explanation in a similar diagnosis scenario. Results: Results show that explanation helps improve satisfaction measures during the critical re-diagnosis period but had little effect before re-diagnosis (when initial treatment was taking place) or after (when an alternate diagnosis resolved the case successfully). Furthermore, initial "global" explanations about the process had no impact on immediate satisfaction but improved later judgments of understanding about the AI. Results of the second experiment show that visual and example-based explanations integrated with rationales had a significantly better impact on patient satisfaction and trust than no explanations, or with text-based rationales alone. As in Experiment 1, these explanations had their effect primarily on immediate measures of satisfaction during the re-diagnosis crisis, with little advantage prior to re-diagnosis or once the diagnosis was successfully resolved. Conclusion: These two studies help us to draw several conclusions about how patient-facing explanatory diagnostic systems may succeed or fail. Based on these studies and the review of the literature, we will provide some design recommendations for the explanations offered for AI systems in the healthcare domain.

Explanatory argumentation in natural language for correct and incorrect medical diagnoses

Argumentation approaches for explanaible AI in medical informatics

CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures

Explanatory argument extraction of correct answers in resident medical exams

What do we need to build explainable AI systems for the medical domain?

A manifesto on explainability for artificial intelligence in medicine

Fine-tuning language model embeddings to reveal domain knowledge: An explainable artificial intelligence perspective on medical decision making

Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems

Improving trust and confidence in medical skin lesion diagnosis through explainable deep learning

Visual explanations for polyp detection: How medical doctors assess intrinsic versus extrinsic explanations

Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting

An Unsupervised Approach to Achieve Supervised-Level Explainability in Healthcare Records

Altruist: Argumentative Explanations through Local Interpretations of Predictive Models

Quantifying decision support level of explainable automatic classification of diagnoses in Spanish medical records

Examining the effect of explanation on satisfaction and trust in AI diagnostic systems

From Explainable to Interpretable Deep Learning for Natural Language Processing in Healthcare: How Far from Reality?

To explain or not to explain?-Artificial intelligence explainability in clinical decision support systems

Beyond Known Reality: Exploiting Counterfactual Explanations for Medical Research

Explainable Artificial Intelligence for Human Decision-Support System in Medical Domain

Integrating Automated Knowledge Extraction with Large Language Models for Explainable Medical Decision-Making