Abstract:The rationale behind a deep learning model's output is often difficult to understand by humans. EXplainable AI (XAI) aims at solving this by developing methods that improve interpretability and explainability of machine learning models. Reliable evaluation metrics are needed to assess and compare different XAI methods. We propose a novel evaluation approach for benchmarking state-of-the-art XAI attribution methods. Our proposal consists of a synthetic classification model accompanied by its derived ground truth explanations allowing high precision representation of input nodes contributions. We also propose new high-fidelity metrics to quantify the difference between explanations of the investigated XAI method and those derived from the synthetic model. Our metrics allow assessment of explanations in terms of precision and recall separately. Also, we propose metrics to independently evaluate negative or positive contributions of inputs. Our proposal provides deeper insights into XAI methods output. We investigate our proposal by constructing a synthetic convolutional image classification model and benchmarking several widely used XAI attribution methods using our evaluation approach. We compare our results with established prior XAI evaluation metrics. By deriving the ground truth directly from the constructed model in our method, we ensure the absence of bias, e.g., subjective either based on the training set. Our experimental results provide novel insights into the performance of Guided-Backprop and Smoothgrad XAI methods that are widely in use. Both have good precision and recall scores among positively contributing pixels (0.7, 0.76 and 0.7, 0.77, respectively), but poor precision scores among negatively contributing pixels (0.44, 0.61 and 0.47, 0.75, resp.). The recall scores in the latter case remain close. We show that our metrics are among the fastest in terms of execution time.

Quantifying Explainability in Outcome-Oriented Predictive Process Monitoring

Which Neural Network Makes More Explainable Decisions? an Approach Towards Measuring Explainability

Explainability in Process Outcome Prediction: Guidelines to Obtain Interpretable and Faithful Models

Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models

Evaluating Explainability in Machine Learning Predictions through Explainer-Agnostic Metrics

Explain To Decide: A Human-Centric Review on the Role of Explainable Artificial Intelligence in AI-assisted Decision Making

Explainable AI without Interpretable Model

Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms

Explaining Explanations: An Overview of Interpretability of Machine Learning

Explainable AI Enabled Inspection of Business Process Prediction Models

EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications

Evaluating Explainable Methods for Predictive Process Analytics: A Functionally-Grounded Approach

Precise Benchmarking of Explainable AI Attribution Methods

OpenXAI: Towards a Transparent Evaluation of Model Explanations

EXACT: Towards a platform for empirically benchmarking Machine Learning model explanation methods

Explainable AI: A Review of Machine Learning Interpretability Methods

Predictability and Comprehensibility in Post-Hoc XAI Methods: A User-Centered Analysis

Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence

Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanations

Black Box Model Explanations and the Human Interpretability Expectations -- An Analysis in the Context of Homicide Prediction

Explainable AI in Credit Risk Management