Abstract:The black-box nature of machine learning models limits their use in case-critical applications, raising faithful and ethical concerns that lead to trust crises. One possible way to mitigate this issue is to understand how a (mispredicted) decision is carved out from the decision boundary. This paper presents a human-in-the-loop approach to explain machine learning models using verbatim neighborhood manifestation. Contrary to most of the current eXplainable Artificial Intelligence (XAI) systems, which provide hit-or-miss approximate explanations, our approach generates the local decision boundary of the given instance and enables human intelligence to conclude the model behavior. Our method can be divided into three stages: 1) a neighborhood generation stage, which generates instances based on the given sample; 2) a classification stage, which yields classifications on the generated instances to carve out the local decision boundary and delineate the model behavior; and 3) a human-in-the-loop stage, which involves human to refine and explore the neighborhood of interest. In the generation stage, a generative model is used to generate the plausible synthetic neighbors around the given instance. After the classification stage, the classified neighbor instances provide a multifaceted understanding of the model behavior. Three intervention points are provided in the human-in-the-loop stage, enabling humans to leverage their own intelligence to interpret the model behavior. Several experiments on two datasets are conducted, and the experimental results demonstrate the potential of our proposed approach for boosting human understanding of the complex machine learning model.

The Intriguing Properties of Model Explanations

Accurate and Intuitive Contextual Explanations using Linear Model Trees

Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction

On the Relationship Between Explanation and Prediction: A Causal View

Explaining Explanations in AI

Learning Model Agnostic Explanations via Constraint Programming

Selective Explanations

Towards a Unified Framework for Evaluating Explanations

Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations

Respect the model: Fine-grained and Robust Explanation with Sharing Ratio Decomposition

An Interpretable Neural Network Model Through Piecewise Linear Approximation

Human-in-the-Loop Model Explanation via Verbatim Boundary Identification in Generated Neighborhoods

Causality-Aware Local Interpretable Model-Agnostic Explanations

Exploring local explanations of nonlinear models using animated linear projections

Dissenting Explanations: Leveraging Disagreement to Reduce Model Overreliance

CXPlain: Causal Explanations for Model Interpretation under Uncertainty

CUBE: Causal Intervention-Based Counterfactual Explanation for Prediction Models

Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models

Considerations When Learning Additive Explanations for Black-Box Models

Minimalistic Explanations: Capturing the Essence of Decisions

Can I Trust the Explanations? Investigating Explainable Machine Learning Methods for Monotonic Models