Bayesian Reasoning for Physics Informed Neural Networks

Krzysztof M. Graczyk,Kornel Witkowski

2024-04-29

Abstract:We present the application of the physics-informed neural network (PINN) approach in Bayesian formulation. We have adopted the Bayesian neural network framework to obtain posterior densities from Laplace approximation. For each model or fit, the evidence is computed, which is a measure that classifies the hypothesis. The optimal solution is the one with the highest value of evidence. We have proposed a modification of the Bayesian algorithm to obtain hyperparameters of the model. We have shown that within the Bayesian framework, one can obtain the relative weights between the boundary and equation contributions to the total loss. Presented method leads to predictions comparable to those obtained by sampling from the posterior distribution within the Hybrid Monte Carlo algorithm (HMC). We have solved heat, wave, and Burger's equations, and the results obtained are in agreement with the exact solutions, demonstrating the effectiveness of our approach. In Burger's equation problem, we have demonstrated that the framework can combine information from differential equations and potential measurements. All solutions are provided with uncertainties (induced by the model's parameter dependence) computed within the Bayesian framework.

Computational Physics,Machine Learning,Fluid Dynamics

What problem does this paper attempt to address?

The paper mainly discusses how to combine the Physics-Informed Neural Network (PINN) method with the Bayesian framework to solve partial differential equation (PDE) problems. The authors propose a modified Bayesian algorithm for obtaining the hyperparameters of the model and demonstrate that this method can achieve predictions comparable to those obtained by the Hamiltonian Monte Carlo (HMC) algorithm. In this way, they solve heat equation, wave equation, and Burgers' equation, and the results are consistent with the exact solutions, proving the effectiveness of the method. The core issue of the paper is to improve the performance and reliability of the PINN method, particularly through Bayesian statistics to handle model uncertainty, hyperparameter selection, and the influence of boundary conditions. Traditional PINN methods may limit their generality due to encoding initial or boundary conditions, while the new method allows these conditions to be introduced in the loss function as relative weights, resulting in a smoother training process. Under the Bayesian framework, the model evidence is used as a measure of hypothesis classification, and the optimal solution corresponds to the model with the highest evidence value. Through Laplace approximation, the posterior probability density can be obtained to estimate model parameters and prediction uncertainty. The paper also discusses how to adjust the relative weights of different loss terms to optimize the influence of boundary terms on the solution. The organization of the paper is as follows: briefly introduce PINN after the introduction, then review the Bayesian framework, followed by a comparison with the HMC method and the solution of heat, wave, and Burgers' equations, and finally, summarize the research. The entire study aims to improve the PINN method using Bayesian inference to enhance its performance in solving PDE problems.

Bayesian Reasoning for Physics Informed Neural Networks

B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data

Efficient Bayesian inference using physics-informed invertible neural networks for inverse problems

Bayesian Deep Learning Framework for Uncertainty Quantification in Stochastic Partial Differential Equations

Efficient Bayesian Physics Informed Neural Networks for inverse problems via Ensemble Kalman Inversion

Error-Aware B-PINNs: Improving Uncertainty Quantification in Bayesian Physics-Informed Neural Networks

Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation

Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley-Leverett problem

Error estimates of physics-informed neural networks for approximating Boltzmann equation

Correcting model misspecification in physics-informed neural networks (PINNs)

Recipes for when physics fails: recovering robust learning of physics informed neural networks

An Investigation of Physics Informed Neural Networks to solve the Poisson-Boltzmann Equation in Molecular Electrostatics

Learning functional priors and posteriors from data and physics

On the estimation rate of Bayesian PINN for inverse problems

A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations

Bayesian Physics Informed Neural Networks for Data Assimilation and Spatio-Temporal Modelling of Wildfires

Deep NURBS -- Admissible Physics-informed Neural Networks

Physics-Informed Neural Networks with Adaptive Localized Artificial Viscosity

Physics-informed Neural Networks with Unknown Measurement Noise

Machine Learning Through Physics–Informed Neural Networks: Progress and Challenges