Abstract:The field of radiology imaging has experienced a remarkable increase in using of deep learning (DL) algorithms to support diagnostic and treatment decisions. This rise has led to the development of Explainable AI (XAI) system to improve the transparency and trust of complex DL methods. However, XAI systems face challenges in gaining acceptance within the healthcare sector, mainly due to technical hurdles in utilizing these systems in practice and the lack of human-centered evaluation/validation. In this study, we focus on visual XAI systems applied to DL-enabled diagnostic system in chest radiography. In particular, we conduct a user study to evaluate two prominent visual XAI techniques from the human perspective. To this end, we created two clinical scenarios for diagnosing pneumonia and COVID-19 using DL techniques applied to chest X-ray and CT scans. The achieved accuracy rates were 90% for pneumonia and 98% for COVID-19. Subsequently, we employed two well-known XAI methods, Grad-CAM (Gradient-weighted Class Activation Mapping) and LIME (Local Interpretable Model-agnostic Explanations), to generate visual explanations elucidating the AI decision-making process. The visual explainability results were shared through a user study, undergoing evaluation by medical professionals in terms of clinical relevance, coherency, and user trust. In general, participants expressed a positive perception of the use of XAI systems in chest radiography. However, there was a noticeable lack of awareness regarding their value and practical aspects. Regarding preferences, Grad-CAM showed superior performance over LIME in terms of coherency and trust, although concerns were raised about its clinical usability. Our findings highlight key user-driven explainability requirements, emphasizing the importance of multi-modal explainability and the necessity to increase awareness of XAI systems among medical practitioners. Inclusive design was also identified as a crucial need to ensure better alignment of these systems with user needs.

Visual Analysis of Agricultural Workers using Explainable Artificial Intelligence (XAI) on Class Activation Map (CAM) with Characteristic Point Data Output from OpenCV-based Analysis

Analyses of Diverse Agricultural Worker Data with Explainable Artificial Intelligence: XAI based on SHAP, LIME, and LightGBM

Toward Human-centered XAI in Practice: A Survey.

Exclusive Feature Constrained Class Activation Mapping for Better Visual Explanation.

VisionCam: A Comprehensive XAI Toolkit for Interpreting Image-Based Deep Learning Models

Analyzing and assessing explainable AI models for smart agriculture environments

Video Surveillance System Incorporating Expert Decision-making Process: A Case Study on Detecting Calving Signs in Cattle

CAManim: Animating end-to-end network activation maps

Agricultural Worker Tracking and Gesture Recognition System for Remote Assistance

Grad-CAM-Based Explainable Artificial Intelligence Related to Medical Text Processing

Evaluating Explainable Artificial Intelligence (XAI) techniques in chest radiology imaging through a human-centered Lens

Enhanced Infield Agriculture with Interpretable Machine Learning Approaches for Crop Classification

From Movements to Metrics: Evaluating Explainable AI Methods in Skeleton-Based Human Activity Recognition

A Video-Based Activity Classification of Human Pickers in Agriculture

XEdgeAI: A Human-centered Industrial Inspection Framework with Data-centric Explainable Edge AI Approach

Evaluating Explainable AI Methods in Deep Learning Models for Early Detection of Cerebral Palsy

A Study on the Reliability of Visual XAI Methods for X-Ray Images

XAI-Enhanced Semantic Segmentation Models for Visual Quality Inspection

HayCAMJ: A new method to uncover the importance of main filter for small objects in explainable artificial intelligence

OAK4XAI: Model towards Out-Of-Box eXplainable Artificial Intelligence for Digital Agriculture