Abstract:Locally interpretable model agnostic explanations (LIME) method is one of the most popular methods used to explain black-box models at a per example level. Although many variants have been proposed, few provide a simple way to produce high fidelity explanations that are also stable and intuitive. In this work, we provide a novel perspective by proposing a model agnostic local explanation method inspired by the invariant risk minimization (IRM) principle -- originally proposed for (global) out-of-distribution generalization -- to provide such high fidelity explanations that are also stable and unidirectional across nearby examples. Our method is based on a game theoretic formulation where we theoretically show that our approach has a strong tendency to eliminate features where the gradient of the black-box function abruptly changes sign in the locality of the example we want to explain, while in other cases it is more careful and will choose a more conservative (feature) attribution, a behavior which can be highly desirable for recourse. Empirically, we show on tabular, image and text data that the quality of our explanations with neighborhoods formed using random perturbations are much better than LIME and in some cases even comparable to other methods that use realistic neighbors sampled from the data manifold. This is desirable given that learning a manifold to either create realistic neighbors or to project explanations is typically expensive or may even be impossible. Moreover, our algorithm is simple and efficient to train, and can ascertain stable input features for local decisions of a black-box without access to side information such as a (partial) causal graph as has been seen in some recent works.

EBLIME: Enhanced Bayesian Local Interpretable Model-agnostic Explanations

BMB-LIME: LIME with modeling local nonlinearity and uncertainty in explainability

BELLA: Black box model Explanations by Local Linear Approximations

Locally Invariant Explanations: Towards Stable and Unidirectional Explanations through Local Invariant Learning

Local Interpretable Model-agnostic Explanations of Bayesian Predictive Models via Kullback-Leibler Projections

Explaining Black-box Models for Biomedical Text Classification

Benchmarking and survey of explanation methods for black box models

Evaluating Local Explanations using White-box Models

An Interpretable Probabilistic Approach for Demystifying Black-box Predictive Models

Selective Explanations

Local Interpretable Model Agnostic Shap Explanations for machine learning models

LaPLACE: Probabilistic Local Model-Agnostic Causal Explanations

RbX: Region-based explanations of prediction models

An Extension of LIME with Improvement of Interpretability and Fidelity

Probabilistic Lipschitzness and the Stable Rank for Comparing Explanation Models

Constraint-Driven Explanations for Black-Box ML Models

"Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations

Explanations of Black-Box Models based on Directional Feature Interactions

Model Agnostic Multilevel Explanations

Local Rule-Based Explanations of Black Box Decision Systems

Explainability as statistical inference