Abstract:The non-transparency of artificial intelligence (AI) systems, particularly in deep learning (DL), poses significant challenges to their comprehensibility and trustworthiness. This study aims to enhance the explainability of DL models through visual analytics (VA) and human-in-the-loop (HITL) principles, making these systems more transparent and understandable to end users. In this work, we propose a novel approach that utilizes a transition matrix to interpret results from DL models through more comprehensible machine learning (ML) models. The methodology involves constructing a transition matrix between the feature spaces of DL and ML models as formal and mental models, respectively, improving the explainability for classification tasks. We validated our approach with computational experiments on the MNIST, FNC-1, and Iris datasets using a qualitative and quantitative comparison criterion, that is, how different the results obtained by our approach are from the ground truth of the training and testing samples. The proposed approach significantly enhanced model clarity and understanding in the MNIST dataset, with SSIM and PSNR values of 0.697 and 17.94, respectively, showcasing high-fidelity reconstructions. Moreover, achieving an F1m score of 77.76% and a weighted accuracy of 89.38%, our approach proved its effectiveness in stance detection with the FNC-1 dataset, complemented by its ability to explain key textual nuances. For the Iris dataset, the separating hyperplane constructed based on the proposed approach allowed for enhancing classification accuracy. Overall, using VA, HITL principles, and a transition matrix, our approach significantly improves the explainability of DL models without compromising their performance, marking a step forward in developing more transparent and trustworthy AI systems.

Human-inspired Explanations for Vision Transformers and Convolutional Neural Networks

A Pixel-Level Explainable Approach of Convolutional Neural Networks and Its Application

Explainability of Vision Transformers: A Comprehensive Review and New Perspectives

On the Faithfulness of Vision Transformer Explanations

T-TAME: Trainable Attention Mechanism for Explaining Convolutional Networks and Vision Transformers

C2F-Explainer: Explaining Transformers Better Through a Coarse-to-Fine Strategy

Embedding deep networks into visual explanations

Spatial-temporal Concept Based Explanation of 3D ConvNets.

ViT-ReciproCAM: Gradient and Attention-Free Visual Explanations for Vision Transformer

Faithful Counterfactual Visual Explanations (FCVE)

Visual correspondence-based explanations improve AI robustness and human-AI team accuracy

Explainable Emotion Decoding for Human and Computer Vision

Inpainting the Gaps: A Novel Framework for Evaluating Explanation Methods in Vision Transformers

Foiling Explanations in Deep Neural Networks

Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformer

Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations

TVE: Learning Meta-attribution for Transferable Vision Explainer

R-Cut: Enhancing Explainability in Vision Transformers with Relationship Weighted Out and Cut

Explainable Vision Transformers for Vein Biometric Recognition

ViT-CX: Causal Explanation of Vision Transformers

Explainable Deep Learning: A Visual Analytics Approach with Transition Matrices