Abstract:Due to their power and ease of use, tree-based machine learning models, such as random forests and gradient-boosted tree ensembles, have become very popular. To interpret them, local feature attributions based on marginal expectations, e.g. marginal (interventional) Shapley, Owen or Banzhaf values, may be employed. Such methods are true to the model and implementation invariant, i.e. dependent only on the input-output function of the model. We contrast this with the popular TreeSHAP algorithm by presenting two (statistically similar) decision trees that compute the exact same function for which the "path-dependent" TreeSHAP yields different rankings of features, whereas the marginal Shapley values coincide. Furthermore, we discuss how the internal structure of tree-based models may be leveraged to help with computing their marginal feature attributions according to a linear game value. One important observation is that these are simple (piecewise-constant) functions with respect to a certain grid partition of the input space determined by the trained model. Another crucial observation, showcased by experiments with XGBoost, LightGBM and CatBoost libraries, is that only a portion of all features appears in a tree from the ensemble. Thus, the complexity of computing marginal Shapley (or Owen or Banzhaf) feature attributions may be reduced. This remains valid for a broader class of game values which we shall axiomatically characterize. A prime example is the case of CatBoost models where the trees are oblivious (symmetric) and the number of features in each of them is no larger than the depth. We exploit the symmetry to derive an explicit formula, with improved complexity and only in terms of the internal model parameters, for marginal Shapley (and Banzhaf and Owen) values of CatBoost models. This results in a fast, accurate algorithm for estimating these feature attributions.

Interventional SHAP Values and Interaction Values for Piecewise Linear Regression Trees

Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles

Consistent Individualized Feature Attribution for Tree Ensembles

A $k$-additive Choquet integral-based approach to approximate the SHAP values for local interpretability in machine learning

Interpretation with Baseline Shapley Value for Feature Groups on Tree Models

On marginal feature attributions of tree-based models

Amortized SHAP values via sparse Fourier function approximation

Improving the Sampling Strategy in KernelSHAP

Analyze Additive and Interaction Effects via Collaborative Trees

GPUTreeShap: Massively Parallel Exact Calculation of SHAP Scores for Tree Ensembles

A comparative study of methods for estimating model-agnostic Shapley value explanations

Explaining the Model and Feature Dependencies by Decomposition of the Shapley Value

Fast Estimation of Partial Dependence Functions using Trees

Unifying local and global model explanations by functional decomposition of low dimensional structures

On the tractability of SHAP explanations under Markovian distributions

Succint Interaction-Aware Explanations

SHAP-IQ: Unified Approximation of any-order Shapley Interactions

Feature-Specific Coefficients of Determination in Tree Ensembles

KernelSHAP-IQ: Weighted Least-Square Optimization for Shapley Interactions

shapiq: Shapley Interactions for Machine Learning

A Comparative Study of Methods for Estimating Conditional Shapley Values and When to Use Them