Abstract:Understanding the interpretation of machine learning (ML) models has been of paramount importance when making decisions with societal impacts such as transport control, financial activities, and medical diagnosis. While current model interpretation methodologies focus on using locally linear functions to approximate the models or creating self-explanatory models that give explanations to each input instance, they do not focus on model interpretation at the subpopulation level, which is the understanding of model interpretations across different subset aggregations in a dataset. To address the challenges of providing explanations of an ML model across the whole dataset, we propose SUBPLEX, a visual analytics system to help users understand black-box model explanations with subpopulation visual analysis. SUBPLEX is designed through an iterative design process with machine learning researchers to address three usage scenarios of real-life machine learning tasks: model debugging, feature selection, and bias detection. The system applies novel subpopulation analysis on ML model explanations and interactive visualization to explore the explanations on a dataset with different levels of granularity. Based on the system, we conduct user evaluation to assess how understanding the interpretation at a subpopulation level influences the sense-making process of interpreting ML models from a user's perspective. Our results suggest that by providing model explanations for different groups of data, SUBPLEX encourages users to generate more ingenious ideas to enrich the interpretations. It also helps users to acquire a tight integration between programming workflow and visual analytics workflow. Last but not least, we summarize the considerations observed in applying visualization to machine learning interpretations.

CoSP: Co-Selection Pick for a Global Explainability of Black Box Machine Learning Models

Constraint-Driven Explanations for Black-Box ML Models

Model Agnostic Multilevel Explanations

Local Interpretable Model Agnostic Shap Explanations for machine learning models

LaPLACE: Probabilistic Local Model-Agnostic Causal Explanations

Learning Model Agnostic Explanations via Constraint Programming

Selective Explanations

SynthTree: Co-supervised Local Model Synthesis for Explainable Prediction

Benchmarking and survey of explanation methods for black box models

Considerations When Learning Additive Explanations for Black-Box Models

Quantifying Explainability in Outcome-Oriented Predictive Process Monitoring

Multi-Objective Counterfactual Explanations

Dissenting Explanations: Leveraging Disagreement to Reduce Model Overreliance

Locally Invariant Explanations: Towards Stable and Unidirectional Explanations through Local Invariant Learning

Learning Groupwise Explanations for Black-Box Models.

An Extension of LIME with Improvement of Interpretability and Fidelity

Explaining black boxes with a SMILE: Statistical Model-agnostic Interpretability with Local Explanations

Local Rule-Based Explanations of Black Box Decision Systems

SUBPLEX: Towards a Better Understanding of Black Box Model Explanations at the Subpopulation Level

MASALA: Model-Agnostic Surrogate Explanations by Locality Adaptation

GLEAMS: Bridging the Gap Between Local and Global Explanations