Abstract:Symbolic regression with polynomial neural networks and polynomial neural ordinary differential equations (ODEs) are two recent and powerful approaches for equation recovery of many science and engineering problems. However, these methods provide point estimates for the model parameters and are currently unable to accommodate noisy data. We address this challenge by developing and validating the following Bayesian inference methods: the Laplace approximation, Markov Chain Monte Carlo (MCMC) sampling methods, and variational inference. We have found the Laplace approximation to be the best method for this class of problems. Our work can be easily extended to the broader class of symbolic neural networks to which the polynomial neural network belongs. Polynomial neural ordinary differential equations (ODEs) are a recent approach for symbolic regression of dynamical systems governed by polynomials. However, they are limited in that they provide maximum likelihood point estimates of the model parameters. The domain expert using system identification often desires a specified level of confidence or range of parameter values that best fit the data. In this work, we use Bayesian inference to provide posterior probability distributions of the parameters in polynomial neural ODEs. To date, there are no studies that attempt to identify the best Bayesian inference method for neural ODEs and symbolic neural ODEs. To address this need, we explore and compare three different approaches for estimating the posterior distributions of weights and biases of the polynomial neural network: the Laplace approximation, Markov Chain Monte Carlo (MCMC) sampling, and variational inference. We have found the Laplace approximation to be the best method for this class of problems. We have also developed lightweight JAX code to estimate posterior probability distributions using the Laplace approximation.

Bayesian inference of covariate-parameter relationships for population modelling

Bayesian variable selection in linear regression models with instrumental variables

A Bayesian Collocation Integral Method for Parameter Estimation in Ordinary Differential Equations

High-dimensional Bayesian Parameter Estimation: Case Study for a Model of JAK2/STAT5 Signaling.

Optimal design of large-scale nonlinear Bayesian inverse problems under model uncertainty

Bayesian estimation for longitudinal data in a joint model with HPCs

Complex-valued Bayesian parameter estimation via Markov chain Monte Carlo

Bayesian inversion of a diffusion model with application to biology

Bayesian Model Selection for High-Dimensional Ising Models, With Applications to Educational Data

Bayesian Nonparametric Covariance Regression

Bayesian Neural Ordinary Differential Equations

Efficient Amortised Bayesian Inference for Hierarchical and Nonlinear Dynamical Systems

Bayesian Numerical Methods for Nonlinear Partial Differential Equations

Parameter estimation of ODE's via nonparametric estimators

Numerical Method for Parameter Inference of Nonlinear ODEs with Partial Observations

A Fast and Scalable Method for A-Optimal Design of Experiments for Infinite-dimensional Bayesian Nonlinear Inverse Problems

Choosing observation operators to mitigate model error in Bayesian inverse problems

Non-intrusive optimal experimental design for large-scale nonlinear Bayesian inverse problems using a Bayesian approximation error approach

Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo

A Bayesian approach to parameter identification with an application to Turing systems

Bayesian polynomial neural networks and polynomial neural ordinary differential equations