Abstract:There has been a significant surge of interest recently around the concept of explainable artificial intelligence (XAI), where the goal is to produce an interpretation for a decision made by a machine learning algorithm. Of particular interest is the interpretation of how deep neural networks make decisions, given the complexity and `black box' nature of such networks. Given the infancy of the field, there has been very limited exploration into the assessment of the performance of explainability methods, with most evaluations centered around subjective visual interpretation of the produced interpretations. In this study, we explore a more machine-centric strategy for quantifying the performance of explainability methods on deep neural networks via the notion of decision-making impact analysis. We introduce two quantitative performance metrics: i) Impact Score, which assesses the percentage of critical factors with either strong confidence reduction impact or decision changing impact, and ii) Impact Coverage, which assesses the percentage coverage of adversarially impacted factors in the input. A comprehensive analysis using this approach was conducted on several state-of-the-art explainability methods (LIME, SHAP, Expected Gradients, GSInquire) on a ResNet-50 deep convolutional neural network using a subset of ImageNet for the task of image classification. Experimental results show that the critical regions identified by LIME within the tested images had the lowest impact on the decision-making process of the network (~38%), with progressive increase in decision-making impact for SHAP (~44%), Expected Gradients (~51%), and GSInquire (~76%). While by no means perfect, the hope is that the proposed machine-centric strategy helps push the conversation forward towards better metrics for evaluating explainability methods and improve trust in deep neural networks.

On Pitfalls of $\textit{RemOve-And-Retrain}$: Data Processing Inequality Perspective

Geometric Remove-and-Retrain (GOAR): Coordinate-Invariant eXplainable AI Assessment

A comprehensive and reliable feature attribution method: Double-sided remove and reconstruct (DoRaR)

ROBY: Evaluating the adversarial robustness of a deep model by its decision boundaries

Relevance-aware Algorithmic Recourse

Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learning

DORO: Distributional and Outlier Robust Optimization

On the Fairness ROAD: Robust Optimization for Adversarial Debiasing

Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms

AROhI: An Interactive Tool for Estimating ROI of Data Analytics

Too Good To Be True: performance overestimation in (re)current practices for Human Activity Recognition

Rethinking Robustness of Model Attributions

Subtle Data Crimes: Naively training machine learning algorithms could lead to overly-optimistic results

Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification?

Inference-Time Rule Eraser: Fair Recognition via Distilling and Removing Biased Rules

DatasetEquity: Are All Samples Created Equal? In The Quest For Equity Within Datasets

On the Evaluation Consistency of Attribution-based Explanations

Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networks

RobOT: Robustness-Oriented Testing for Deep Learning Systems

Feature Perturbation Augmentation for Reliable Evaluation of Importance Estimators in Neural Networks