Abstract:A key challenge for decision makers when incorporating black box machine learned models into practice is being able to understand the predictions provided by these models. One set of methods proposed to address this challenge is that of training surrogate explainer models which approximate how the more complex model is computing its predictions. Explainer methods are generally classified as either local or global explainers depending on what portion of the data space they are purported to explain. The improved coverage of global explainers usually comes at the expense of explainer fidelity (i.e., how well the explainer's predictions match that of the black box model). One way of trading off the advantages of both approaches is to aggregate several local explainers into a single explainer model with improved coverage. However, the problem of aggregating these local explainers is computationally challenging, and existing methods only use heuristics to form these aggregations. In this paper, we propose a local explainer aggregation method which selects local explainers using non-convex optimization. In contrast to other heuristic methods, we use an integer optimization framework to combine local explainers into a near-global aggregate explainer. Our framework allows a decision-maker to directly tradeoff coverage and fidelity of the resulting aggregation through the parameters of the optimization problem. We also propose a novel local explainer algorithm based on information filtering. We evaluate our algorithmic framework on two healthcare datasets: the Parkinson's Progression Marker Initiative (PPMI) data set and a geriatric mobility dataset from the UCI machine learning repository. Our choice of these healthcare-related datasets is motivated by the anticipated need for explainable precision medicine. We find that our method outperforms existing local explainer aggregation methods in terms of both fidelity and coverage of classification. It also improves on fidelity over existing global explainer methods, particularly in multi-class settings, where state-of-the-art methods achieve 70% and ours achieves 90%.

Fuzzy Rule-Based Local Surrogate Models for Black-Box Model Explanation

Local Rule-Based Explanations of Black Box Decision Systems

Human-in-the-Loop Model Explanation via Verbatim Boundary Identification in Generated Neighborhoods

"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations

Local Interpretations for Explainable Natural Language Processing: A Survey

An Empirical Comparison of Interpretable Models to Post-Hoc Explanations

Surrogate Model-Based Explainability Methods for Point Cloud NNs

A Survey on the Explainability of Supervised Machine Learning

Explaining black boxes with a SMILE: Statistical Model-agnostic Interpretability with Local Explanations

Evaluating Local Explanations using White-box Models

Fuzzy-clustering and fuzzy network based interpretable fuzzy model for prediction

SUBPLEX: Towards a Better Understanding of Black Box Model Explanations at the Subpopulation Level

BELLA: Black box model Explanations by Local Linear Approximations

How to Build Self-Explaining Fuzzy Systems: From Interpretability to Explainability

Black Box Model Explanations and the Human Interpretability Expectations -- An Analysis in the Context of Homicide Prediction

Optimal Local Explainer Aggregation for Interpretable Prediction

Local Interpretable Model Agnostic Shap Explanations for machine learning models

Explaining Black-box Model Predictions via Two-level Nested Feature Attributions with Consistency Property

A Survey of Methods for Explaining Black Box Models

Benchmarking and survey of explanation methods for black box models