Abstract:The field of radiology imaging has experienced a remarkable increase in using of deep learning (DL) algorithms to support diagnostic and treatment decisions. This rise has led to the development of Explainable AI (XAI) system to improve the transparency and trust of complex DL methods. However, XAI systems face challenges in gaining acceptance within the healthcare sector, mainly due to technical hurdles in utilizing these systems in practice and the lack of human-centered evaluation/validation. In this study, we focus on visual XAI systems applied to DL-enabled diagnostic system in chest radiography. In particular, we conduct a user study to evaluate two prominent visual XAI techniques from the human perspective. To this end, we created two clinical scenarios for diagnosing pneumonia and COVID-19 using DL techniques applied to chest X-ray and CT scans. The achieved accuracy rates were 90% for pneumonia and 98% for COVID-19. Subsequently, we employed two well-known XAI methods, Grad-CAM (Gradient-weighted Class Activation Mapping) and LIME (Local Interpretable Model-agnostic Explanations), to generate visual explanations elucidating the AI decision-making process. The visual explainability results were shared through a user study, undergoing evaluation by medical professionals in terms of clinical relevance, coherency, and user trust. In general, participants expressed a positive perception of the use of XAI systems in chest radiography. However, there was a noticeable lack of awareness regarding their value and practical aspects. Regarding preferences, Grad-CAM showed superior performance over LIME in terms of coherency and trust, although concerns were raised about its clinical usability. Our findings highlight key user-driven explainability requirements, emphasizing the importance of multi-modal explainability and the necessity to increase awareness of XAI systems among medical practitioners. Inclusive design was also identified as a crucial need to ensure better alignment of these systems with user needs.

Explainable artificial intelligence for medical imaging: Review and experiments with infrared breast images

Analysis of Explainable Artificial Intelligence Methods on Medical Image Classification

Explainable Artificial Intelligence for Human Decision-Support System in Medical Domain

Explainable Artificial Intelligence for Medical Applications: A Review

Quantitative evaluation of Saliency-Based Explainable artificial intelligence (XAI) methods in Deep Learning-Based mammogram analysis

Evaluating Explainable Artificial Intelligence (XAI) techniques in chest radiology imaging through a human-centered Lens

Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks

Unveiling the black box: A systematic review of Explainable Artificial Intelligence in medical image analysis

Breast Cancer Diagnosis: A Comprehensive Exploration of Explainable Artificial Intelligence (XAI) Techniques

Applications of Explainable Artificial Intelligence in Diagnosis and Surgery

Explainable Deep Learning Methods in Medical Imaging Diagnosis: A Survey

Explainable machine learning for breast cancer diagnosis from mammography and ultrasound images: a systematic review

Explainable Artificial Intelligence in Retinal Imaging for the detection of Systemic Diseases

Explainable artificial intelligence (XAI) in radiology and nuclear medicine: a literature review

Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems

A scoping review of interpretability and explainability concerning artificial intelligence methods in medical imaging

Transparency of deep neural networks for medical image analysis: A review of interpretability methods

Explainable AI applications in the Medical Domain: a systematic review

Explainable AI And Visual Reasoning: Insights From Radiology

Interpretable Medical Imagery Diagnosis with Self-Attentive Transformers: A Review of Explainable AI for Health Care

Fully Interpretable Deep Learning Model Using IR Thermal Images for Possible Breast Cancer Cases