Abstract:The non-transparency of artificial intelligence (AI) systems, particularly in deep learning (DL), poses significant challenges to their comprehensibility and trustworthiness. This study aims to enhance the explainability of DL models through visual analytics (VA) and human-in-the-loop (HITL) principles, making these systems more transparent and understandable to end users. In this work, we propose a novel approach that utilizes a transition matrix to interpret results from DL models through more comprehensible machine learning (ML) models. The methodology involves constructing a transition matrix between the feature spaces of DL and ML models as formal and mental models, respectively, improving the explainability for classification tasks. We validated our approach with computational experiments on the MNIST, FNC-1, and Iris datasets using a qualitative and quantitative comparison criterion, that is, how different the results obtained by our approach are from the ground truth of the training and testing samples. The proposed approach significantly enhanced model clarity and understanding in the MNIST dataset, with SSIM and PSNR values of 0.697 and 17.94, respectively, showcasing high-fidelity reconstructions. Moreover, achieving an F1m score of 77.76% and a weighted accuracy of 89.38%, our approach proved its effectiveness in stance detection with the FNC-1 dataset, complemented by its ability to explain key textual nuances. For the Iris dataset, the separating hyperplane constructed based on the proposed approach allowed for enhancing classification accuracy. Overall, using VA, HITL principles, and a transition matrix, our approach significantly improves the explainability of DL models without compromising their performance, marking a step forward in developing more transparent and trustworthy AI systems.

Representing visual classification as a linear combination of words

Aligning Human Knowledge with Visual Concepts Towards Explainable Medical Image Classification

Aligning Characteristic Descriptors with Images for Human-Expert-like Explainability

Explaining high-dimensional text classifiers

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models

VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Models.

Envisioning MedCLIP: A Deep Dive into Explainability for Medical Vision-Language Models

Are Explanations Helpful? A Comparative Analysis of Explainability Methods in Skin Lesion Classifiers

Explainable Deep Image Classifiers for Skin Lesion Diagnosis

Explanation Strategies for Image Classification in Humans vs. Current Explainable AI

Seeing in Words: Learning to Classify through Language Bottlenecks

Explainable Deep Learning: A Visual Analytics Approach with Transition Matrices

Exemplars and Counterexemplars Explanations for Image Classifiers, Targeting Skin Lesion Labeling

Explaining Chest X-ray Pathology Models using Textual Concepts

Grad-CAM-Based Explainable Artificial Intelligence Related to Medical Text Processing

Evaluating the Explainability of Attributes and Prototypes for a Medical Classification Model

Sim2Word: Explaining Similarity with Representative Attribute Words via Counterfactual Explanations

Category-Independent Visual Explanation for Medical Deep Network Understanding

Improving Explainability of Disentangled Representations using Multipath-Attribution Mappings

Explainable Deep Classification Models for Domain Generalization