Abstract:There has been a significant surge of interest recently around the concept of explainable artificial intelligence (XAI), where the goal is to produce an interpretation for a decision made by a machine learning algorithm. Of particular interest is the interpretation of how deep neural networks make decisions, given the complexity and `black box' nature of such networks. Given the infancy of the field, there has been very limited exploration into the assessment of the performance of explainability methods, with most evaluations centered around subjective visual interpretation of the produced interpretations. In this study, we explore a more machine-centric strategy for quantifying the performance of explainability methods on deep neural networks via the notion of decision-making impact analysis. We introduce two quantitative performance metrics: i) Impact Score, which assesses the percentage of critical factors with either strong confidence reduction impact or decision changing impact, and ii) Impact Coverage, which assesses the percentage coverage of adversarially impacted factors in the input. A comprehensive analysis using this approach was conducted on several state-of-the-art explainability methods (LIME, SHAP, Expected Gradients, GSInquire) on a ResNet-50 deep convolutional neural network using a subset of ImageNet for the task of image classification. Experimental results show that the critical regions identified by LIME within the tested images had the lowest impact on the decision-making process of the network (~38%), with progressive increase in decision-making impact for SHAP (~44%), Expected Gradients (~51%), and GSInquire (~76%). While by no means perfect, the hope is that the proposed machine-centric strategy helps push the conversation forward towards better metrics for evaluating explainability methods and improve trust in deep neural networks.

Quality at the Tail of Machine Learning Inference

Improving Data-Driven Inferential Sensor Modeling by Industrial Knowledge: A Bayesian Perspective

An Adversarial Bidirectional Serial–parallel LSTM-based QTD Framework for Product Quality Prediction

Variational Inference over Graph: Knowledge Representation for Deep Process Data Analytics

From Data Quality to Model Quality: an Exploratory Study on Deep Learning

Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models

Dual-Track Lifelong Machine Learning-Based Fine-Grained Product Quality Analysis

Studying the Impact of TensorFlow and PyTorch Bindings on Machine Learning Software Quality

Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving

Understanding Time Variations of DNN Inference in Autonomous Driving

Quality In / Quality Out: Data quality more relevant than model choice in anomaly detection with the UGR'16

Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms

Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluation

Data-Quality-Navigated Machine Learning Strategy with Chemical Intuition to Improve Generalization

Adaptive Inference: Theoretical Limits and Unexplored Opportunities

Incorporating Spatio-Temporal Smoothness for Air Quality Inference.

Quality Classified Image Analysis with Application to Face Detection and Recognition

Quality Management of Machine Learning Systems

Timely Communications for Remote Inference

Automated Customization of On-Device Inference for Quality-of-Experience Enhancement