Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation

Yifei Zong,David Barajas-Solano,Alexandre M. Tartakovsky

2024-07-06

Abstract:We propose a randomized physics-informed neural network (PINN) or rPINN method for uncertainty quantification in inverse partial differential equation (PDE) problems with noisy data. This method is used to quantify uncertainty in the inverse PDE PINN solutions. Recently, the Bayesian PINN (BPINN) method was proposed, where the posterior distribution of the PINN parameters was formulated using the Bayes' theorem and sampled using approximate inference methods such as the Hamiltonian Monte Carlo (HMC) and variational inference (VI) methods. In this work, we demonstrate that HMC fails to converge for non-linear inverse PDE problems. As an alternative to HMC, we sample the distribution by solving the stochastic optimization problem obtained by randomizing the PINN loss function. The effectiveness of the rPINN method is tested for linear and non-linear Poisson equations, and the diffusion equation with a high-dimensional space-dependent diffusion coefficient. The rPINN method provides informative distributions for all considered problems. For the linear Poisson equation, HMC and rPINN produce similar distributions, but rPINN is on average 27 times faster than HMC. For the non-linear Poison and diffusion equations, the HMC method fails to converge because a single HMC chain cannot sample multiple modes of the posterior distribution of the PINN parameters in a reasonable amount of time.

Machine Learning

What problem does this paper attempt to address?

The paper aims to address the following issues:

1. **Uncertainty Quantification**: Proposes a randomized physics-informed neural network (rPINN) method to handle uncertainty quantification in inverse partial differential equation (PDE) problems with noisy data.

2. **Bayesian Inference**: By comparing with existing Bayesian PINN (BPINN) methods, particularly Hamiltonian Monte Carlo (HMC) and variational inference (VI), it demonstrates the limitations of these methods in nonlinear inverse PDE problems.

3. **Optimization Method Improvement**: By introducing random noise into the PINN loss function, the optimization problem is transformed into a stochastic optimization problem, and the posterior distribution samples of the parameters are obtained by solving a series of randomized minimization problems.

Specifically, the paper demonstrates the effectiveness of the rPINN method in handling linear and nonlinear Poisson equations as well as high-dimensional diffusion equations. Experimental results show that in the linear Poisson equation, rPINN is approximately 27 times faster than HMC; whereas in the nonlinear Poisson equation and diffusion equation, HMC fails to converge within a reasonable time, while rPINN can provide effective posterior distributions. Additionally, rPINN addresses the convergence issues encountered by traditional BPINN methods in high-dimensional parameter spaces.