Abstract:Shapley value attribution (SVA) is an increasingly popular explainable AI (XAI) method, which quantifies the contribution of each feature to the model's output. However, recent work has shown that most existing methods to implement SVAs have some drawbacks, resulting in biased or unreliable explanations that fail to correctly capture the true intrinsic relationships between features and model outputs. Moreover, the mechanism and consequences of these drawbacks have not been discussed systematically. In this paper, we propose a novel error theoretical analysis framework, in which the explanation errors of SVAs are decomposed into two components: observation bias and structural bias. We further clarify the underlying causes of these two biases and demonstrate that there is a trade-off between them. Based on this error analysis framework, we develop two novel concepts: over-informative and underinformative explanations. We demonstrate how these concepts can be effectively used to understand potential errors of existing SVA methods. In particular, for the widely deployed assumption-based SVAs, we find that they can easily be under-informative due to the distribution drift caused by distributional assumptions. We propose a measurement tool to quantify such a distribution drift. Finally, our experiments illustrate how different existing SVA methods can be over- or under-informative. Our work sheds light on how errors incur in the estimation of SVAs and encourages new less error-prone methods.

Shapley variable importance cloud for interpretable machine learning

Shapley variable importance clouds for interpretable machine learning

Shapley variable importance cloud for machine learning models

Variable importance analysis with interpretable machine learning for fair risk prediction

Shapley Marginal Surplus for Strong Models

Variable Importance Clouds: A Way to Explore Variable Importance for the Set of Good Models

Comparing interpretability and explainability for feature selection

Local Interpretable Model Agnostic Shap Explanations for machine learning models

A Unified Approach to Interpreting Model Predictions

ShapG: new feature importance method based on the Shapley value

Shapley Value: from Cooperative Game to Explainable Artificial Intelligence

Explaining black box decisions by Shapley cohort refinement

Error Analysis of Shapley Value-Based Model Explanations: An Informative Perspective

Manifold-based Shapley explanations for high dimensional correlated features

An empirical study of the effect of background data size on the stability of SHapley Additive exPlanations (SHAP) for deep learning models

L-shapley and c-shapley: Efficient model interpretation for structured data

Explaining Predictive Uncertainty with Information Theoretic Shapley Values

Enhancing the Interpretability of SHAP Values Using Large Language Models

From global to local MDI variable importances for random forests and when they are Shapley values

Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions

Extracting spatial effects from machine learning model using local interpretation method: An example of SHAP and XGBoost