Abstract:Interpretability is crucial for machine learning in many scenarios such as quantitative finance, banking, healthcare, etc. Symbolic regression (SR) is a classic interpretable machine learning method by bridging X and Y using mathematical expressions composed of some basic functions. However, the search space of all possible expressions grows exponentially with the length of the expression, making it infeasible for enumeration. Genetic programming (GP) has been traditionally and commonly used in SR to search for the optimal solution, but it suffers from several limitations, e.g. the difficulty in incorporating prior knowledge; overly-complicated output expression and reduced interpretability etc. To address these issues, we propose a new method to fit SR under a Bayesian framework. Firstly, Bayesian model can naturally incorporate prior knowledge (e.g., preference of basis functions, operators and raw features) to improve the efficiency of fitting SR. Secondly, to improve interpretability of expressions in SR, we aim to capture concise but informative signals. To this end, we assume the expected signal has an additive structure, i.e., a linear combination of several concise expressions, whose complexity is controlled by a well-designed prior distribution. In our setup, each expression is characterized by a symbolic tree, and the proposed SR model could be solved by sampling symbolic trees from the posterior distribution using an efficient Markov chain Monte Carlo (MCMC) algorithm. Finally, compared with GP, the proposed BSR(Bayesian Symbolic Regression) method saves computer memory with no need to keep an updated 'genome pool'. Numerical experiments show that, compared with GP, the solutions of BSR are closer to the ground truth and the expressions are more concise. Meanwhile we find the solution of BSR is robust to hyper-parameter specifications such as the number of trees.

Generalizability Improvement of Interpretable Symbolic Regression Models for Quantitative Structure-Activity Relationships

Bayesian Symbolic Regression

Quantitative Structure–activity Relationship: Promising Advances in Drug Discovery Platforms

Multi-View Symbolic Regression

Comparison and improvement of the predictability and interpretability with ensemble learning models in QSPR applications

A Transformer Model for Symbolic Regression towards Scientific Discovery

Scalable Neural Symbolic Regression using Control Variables

Interpretable scientific discovery with symbolic regression: a review

Sym-Q: Adaptive Symbolic Regression via Sequential Decision-Making

Optimal Piecewise Linear Regression Algorithm for QSAR Modelling

Online Symbolic Regression with Informative Query

Prediction of the Aquatic Toxicity of Aromatic Compounds to Tetrahymena Pyriformis Through Support Vector Regression

An Interpretable Rule Creation Method for Black-Box Models based on Surrogate Trees -- SRules

QSAR Modeling: Where Have You Been? Where Are You Going To?

Class Symbolic Regression: Gotta Fit 'Em All

Interpretability in Symbolic Regression: a benchmark of Explanatory Methods using the Feynman data set

Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl

Information Fusion via Symbolic Regression: A Tutorial in the Context of Human Health

Deep Generative Symbolic Regression

Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients

Regression Methods For Developing Qsar And Qspr Models To Predict Compounds Of Specific Pharmacodynamic, Pharmacokinetic And Toxicological Properties