Abstract:Interpretability is crucial for machine learning in many scenarios such as quantitative finance, banking, healthcare, etc. Symbolic regression (SR) is a classic interpretable machine learning method by bridging X and Y using mathematical expressions composed of some basic functions. However, the search space of all possible expressions grows exponentially with the length of the expression, making it infeasible for enumeration. Genetic programming (GP) has been traditionally and commonly used in SR to search for the optimal solution, but it suffers from several limitations, e.g. the difficulty in incorporating prior knowledge; overly-complicated output expression and reduced interpretability etc. To address these issues, we propose a new method to fit SR under a Bayesian framework. Firstly, Bayesian model can naturally incorporate prior knowledge (e.g., preference of basis functions, operators and raw features) to improve the efficiency of fitting SR. Secondly, to improve interpretability of expressions in SR, we aim to capture concise but informative signals. To this end, we assume the expected signal has an additive structure, i.e., a linear combination of several concise expressions, whose complexity is controlled by a well-designed prior distribution. In our setup, each expression is characterized by a symbolic tree, and the proposed SR model could be solved by sampling symbolic trees from the posterior distribution using an efficient Markov chain Monte Carlo (MCMC) algorithm. Finally, compared with GP, the proposed BSR(Bayesian Symbolic Regression) method saves computer memory with no need to keep an updated 'genome pool'. Numerical experiments show that, compared with GP, the solutions of BSR are closer to the ground truth and the expressions are more concise. Meanwhile we find the solution of BSR is robust to hyper-parameter specifications such as the number of trees.

A Symbolic Approach to Explaining Bayesian Network Classifiers

Reasoning about Bayesian Network Classifiers

A unified logical framework for explanations in classifier systems

Embedded Bayesian Network Classifiers

Computing Rule-Based Explanations of Machine Learning Classifiers using Knowledge Graphs

Explainable AI for Classification Using Probabilistic Logic Inference

Globally Interpretable Classifiers via Boolean Formulas with Dynamic Propositions

Seeking Interpretability and Explainability in Binary Activated Neural Networks

Viewing the process of generating counterfactuals as a source of knowledge: a new approach for explaining classifiers

Deep Learning for Case-Based Reasoning Through Prototypes: A Neural Network That Explains Its Predictions

Simple and Interpretable Probabilistic Classifiers for Knowledge Graphs

Case-based selection of explanation methods for neural network image classifiers

Explaining Natural Language Processing Classifiers with Occlusion and Language Modeling

Explanation Trees for Causal Bayesian Networks

Minimalistic Explanations: Capturing the Essence of Decisions

Closed-Form Interpretation of Neural Network Classifiers with Symbolic Gradients

Bayesian Symbolic Regression

Axiomatic Characterisations of Sample-based Explainers

Improving Explainability of Softmax Classifiers Using a Prototype-Based Joint Embedding Method

Local Rule-Based Explanations of Black Box Decision Systems