Abstract:The use of sophisticated machine learning models for critical decision making is faced with a challenge that these models are often applied as a "black-box". This has led to an increased interest in interpretable machine learning, where post hoc interpretation presents a useful mechanism for generating interpretations of complex learning models. In this paper, we propose a novel approach underpinned by an extended framework of Bayesian networks for generating post hoc interpretations of a black-box predictive model. The framework supports extracting a Bayesian network as an approximation of the black-box model for a specific prediction. Compared to the existing post hoc interpretation methods, the contribution of our approach is three-fold. Firstly, the extracted Bayesian network, as a probabilistic graphical model, can provide interpretations about not only what input features but also why these features contributed to a prediction. Secondly, for complex decision problems with many features, a Markov blanket can be generated from the extracted Bayesian network to provide interpretations with a focused view on those input features that directly contributed to a prediction. Thirdly, the extracted Bayesian network enables the identification of four different rules which can inform the decision-maker about the confidence level in a prediction, thus helping the decision-maker assess the reliability of predictions learned by a black-box model. We implemented the proposed approach, applied it in the context of two well-known public datasets and analysed the results, which are made available in an open-source repository.

Policy Trees for Prediction: Interpretable and Adaptive Model Selection for Machine Learning

POETREE: Interpretable Policy Learning with Adaptive Decision Trees

Optimizing Interpretable Decision Tree Policies for Reinforcement Learning

Interpretable Local Tree Surrogate Policies

Optimal Decision Tree Policies for Markov Decision Processes

Explainable Adaptive Tree-based Model Selection for Time Series Forecasting

Learning Optimal Prescriptive Trees from Observational Data

Optimal Interpretability-Performance Trade-off of Classification Trees with Black-Box Reinforcement Learning

Decision Trees for Decision-Making under the Predict-then-Optimize Framework

An Interpretable Probabilistic Approach for Demystifying Black-box Predictive Models

Leveraging Model-based Trees as Interpretable Surrogate Models for Model Distillation

Predict-Then-Optimize by Proxy: Learning Joint Models of Prediction and Optimization

Accurate and Intuitive Contextual Explanations using Linear Model Trees

Policy Learning for Many Outcomes of Interest: Combining Optimal Policy Trees with Multi-objective Bayesian Optimisation

SYMPOL: Symbolic Tree-Based On-Policy Reinforcement Learning

Conservative Q-Improvement: Reinforcement Learning for an Interpretable Decision-Tree Policy

Learning Interpretable, High-Performing Policies for Autonomous Driving

Interpretable and Editable Programmatic Tree Policies for Reinforcement Learning

Building more accurate decision trees with the additive tree

Enabling Decision-Making with the Modified Causal Forest: Policy Trees for Treatment Assignment

Building Trees for Probabilistic Prediction via Scoring Rules