Bayesian polynomial neural networks and polynomial neural ordinary differential equations

Colby Fronk,Jaewoong Yun,Prashant Singh,Linda Petzold

2023-08-25

Abstract:Symbolic regression with polynomial neural networks and polynomial neural ordinary differential equations (ODEs) are two recent and powerful approaches for equation recovery of many science and engineering problems. However, these methods provide point estimates for the model parameters and are currently unable to accommodate noisy data. We address this challenge by developing and validating the following Bayesian inference methods: the Laplace approximation, Markov Chain Monte Carlo (MCMC) sampling methods, and variational inference. We have found the Laplace approximation to be the best method for this class of problems. Our work can be easily extended to the broader class of symbolic neural networks to which the polynomial neural network belongs.

Machine Learning,Artificial Intelligence,Computational Engineering, Finance, and Science

What problem does this paper attempt to address?

The paper aims to address the following issues:

1. **Limitations of Existing Methods**: Current Polynomial Neural Networks and Polynomial Neural ODEs can only provide maximum likelihood estimates of parameters when performing symbolic regression, and they cannot handle noisy data or provide uncertainty estimates of the parameters.

2. **Introducing Bayesian Inference**: To overcome the above limitations, the authors developed and validated several Bayesian inference methods, including Laplace Approximation, Markov Chain Monte Carlo (MCMC), and Variational Inference, to obtain the posterior probability distribution of parameters in Polynomial Neural ODEs.

3. **Improving Model Interpretability and Robustness**: Through Bayesian inference methods, the researchers aim to not only obtain the best estimates of model parameters when dealing with real noisy data but also provide confidence intervals or uncertainty estimates of these parameters, thereby enhancing the interpretability and robustness of the model.

4. **Comparing Different Bayesian Inference Methods**: The paper also compares the effectiveness of the aforementioned Bayesian inference methods and ultimately finds that Laplace Approximation is the optimal method for this type of problem, providing corresponding experimental results to support this conclusion.

In summary, the main purpose of this paper is to improve existing Polynomial Neural Networks and Polynomial Neural ODE models by introducing Bayesian inference methods, providing more reliable and interpretable solutions when dealing with complex dynamic systems.