Abstract:Model interpretation, or explanation of a machine learning classifier, aims to extract generalizable knowledge from a trained classifier into a human-understandable format, for various purposes such as model assessment, debugging and trust. From a computaional viewpoint, it is formulated as approximating the target classifier using a simpler interpretable model, such as rule models like a decision set/list/tree. Often, this approximation is handled as standard supervised learning and the only difference is that the labels are provided by the target classifier instead of ground truth. This paradigm is particularly popular because there exists a variety of well-studied supervised algorithms for learning an interpretable classifier. However, we argue that this paradigm is suboptimal for it does not utilize the unique property of the model interpretation problem, that is, the ability to generate synthetic instances and query the target classifier for their labels. We call this the active-query property, suggesting that we should consider model interpretation from an active learning perspective. Following this insight, we argue that the active-query property should be employed when designing a model interpretation algorithm, and that the generation of synthetic instances should be integrated seamlessly with the algorithm that learns the model interpretation. In this paper, we demonstrate that by doing so, it is possible to achieve more faithful interpretation with simpler model complexity. As a technical contribution, we present an active algorithm Active Decision Set Induction (ADS) to learn a decision set, a set of if-else rules, for model interpretation. ADS performs a local search over the space of all decision sets. In every iteration, ADS computes confidence intervals for the value of the objective function of all local actions and utilizes active-query to determine the best one.

Performance and Interpretation of Classification Models

A Theory of Diagnostic Interpretation in Supervised Classification

Methods and Models for Interpretable Linear Classification

A Double Penalty Model for Interpretability

Beyond Average Performance -- exploring regions of deviating performance for black box classification models

Testing the Consistency of Performance Scores Reported for Binary Classification Problems

Making accurate and interpretable treatment decisions for binary outcomes

A Text Classification-Based Approach for Evaluating and Enhancing the Machine Interpretability of Building Codes

Classification performance assessment for imbalanced multiclass data

Sensitivity and Specificity versus Precision and Recall, and Related Dilemmas

A Strategy on Selecting Performance Metrics for Classifier Evaluation.

Performance and Interpretability Comparisons of Supervised Machine Learning Algorithms: An Empirical Study

Personalized Interpretable Classification

Evaluating Machine Learning Models against Clinical Protocols for Enhanced Interpretability and Continuity of Care

Interpretable Classification Models for Recidivism Prediction

Empirical analysis of performance assessment for imbalanced classification

IB-M: A Flexible Framework to Align an Interpretable Model and a Black-box Model

Enhancing Clinical Predictive Modeling through Model Complexity-Driven Class Proportion Tuning for Class Imbalanced Data: An Empirical Study on Opioid Overdose Prediction

Interpretable and explainable machine learning: A methods‐centric overview with concrete examples

A Study of Data Pre-processing Techniques for Imbalanced Biomedical Data Classification

An Active Approach for Model Interpretation