Abstract:Recent advancement in deep-neural network performance led to the development of new state-of-the-art approaches in numerous areas. However, the black-box nature of neural networks often prohibits their use in areas where model explainability and model transparency are crucial. Over the years, researchers proposed many algorithms to aid neural network understanding and provide additional information to the human expert. One of the most popular methods being Layer-Wise Relevance Propagation (LRP). This method assigns local relevance based on the pixel-wise decomposition of nonlinear classifiers. With the rise of attribution method research, there has emerged a pressing need to assess and evaluate their performance. Numerous metrics have been proposed, each assessing an individual property of attribution methods such as faithfulness, robustness or localization. Unfortunately, no single metric is deemed optimal for every case, and researchers often use several metrics to test the quality of the attribution maps. In this work, we address the shortcomings of the current LRP formulations and introduce a novel method for determining the relevance of input neurons through layer-wise relevance propagation. Furthermore, we apply this approach to the recently developed Vision Transformer architecture and evaluate its performance against existing methods on two image classification datasets, namely ImageNet and PascalVOC. Our results clearly demonstrate the advantage of our proposed method. Furthermore, we discuss the insufficiencies of current evaluation metrics for attribution-based explainability and propose a new evaluation metric that combines the notions of faithfulness, robustness and contrastiveness. We utilize this new metric to evaluate the performance of various attribution-based methods. Our code is available at: https://github.com/davor10105/relative-absolute-magnitude-propagation

Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimization

Exclusive Feature Constrained Class Activation Mapping for Better Visual Explanation.

Unsupervised discovery of Interpretable Visual Concepts

Understanding Neural Networks Through Deep Visualization

Feature Accentuation: Revealing 'What' Features Respond to in Natural Images

Visualizing the PHATE of Neural Networks

Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations

DISCOVER: Making Vision Networks Interpretable via Competition and Dissection

Interactive dense pixel visualizations for time series and model attribution explanations

Empowering Low-Light Image Enhancer through Customized Learnable Priors

Parallel Backpropagation for Shared-Feature Visualization

Advancing Attribution-Based Neural Network Explainability through Relative Absolute Magnitude Layer-Wise Relevance Propagation and Multi-Component Evaluation

A walk in the black-box: 3D visualization of large neural networks in virtual reality

DeViL: Decoding Vision features into Language

Restyling Unsupervised Concept Based Interpretable Networks with Generative Models

Brain Decodes Deep Nets

Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain

Neural network interpretability with layer-wise relevance propagation: novel techniques for neuron selection and visualization

A Powerful Generative Model Using Random Weights for the Deep Image Representation