Abstract:Physics Informed Neural Networks (PINNs) have been achieving ever newer feats of solving complicated PDEs numerically while offering an attractive trade-off between accuracy and speed of inference. A particularly challenging aspect of PDEs is that there exist simple PDEs which can evolve into singular solutions in finite time starting from smooth initial conditions. In recent times some striking experiments have suggested that PINNs might be good at even detecting such finite-time blow-ups. In this work, we embark on a program to investigate this stability of PINNs from a rigorous theoretical viewpoint. Firstly, we derive generalization bounds for PINNs for Burgers' PDE, in arbitrary dimensions, under conditions that allow for a finite-time blow-up. Then we demonstrate via experiments that our bounds are significantly correlated to the $\ell_2$-distance of the neurally found surrogate from the true blow-up solution, when computed on sequences of PDEs that are getting increasingly close to a blow-up.

What problem does this paper attempt to address?

This paper explores the performance of Physics Informed Neural Networks (PINNs) in solving the Burgers equation with finite time blow-up phenomenon. The Burgers equation is a partial differential equation commonly used to simulate physical processes such as fluid dynamics. However, under certain conditions, its solution may blow up in finite time, meaning it becomes infinite at some point. Although PINNs have shown excellent results in solving complex dynamical systems such as the Navier-Stokes equation, there is a lack of in-depth theoretical research on their stability and accuracy when approaching blow-up solutions. The paper first reviews the general framework of PINNs and defines the loss function and residual related to the Burgers equation. Then, the authors propose a new risk boundary theorem for the Burgers equation with finite time blow-up. These theorems consider zero viscosity settings and boundary conditions consistent with blow-up situations, and they prove that these boundaries remain effective even as the network width increases, reflecting the advantages of overparameterized networks. In the experimental section, the paper demonstrates a significant correlation between the theoretical error boundary and the L2 distance between the approximate solution found by PINNs and the true solution, especially when approaching the blow-up point. Furthermore, the experiments show that for one-dimensional blow-up cases, the time to solve the neural approximation is almost unaffected by proximity to the blow-up point, and the theoretical boundary decreases with increasing network width. The paper also discusses the inadequacy of existing numerical analysis error boundaries for PINNs training, pointing out the potential instability of PINNs training dynamics and the failure to converge to the correct solution. The paper references related work to analyze failure modes of PINNs, such as gradient distribution inequality issues, and proposes solution strategies. In conclusion, the problem addressed in this paper is how to theoretically analyze the performance of PINNs in handling partial differential equations with potential finite time blow-up and how to experimentally validate the effectiveness of these theoretical boundaries.

Investigating the Ability of PINNs To Solve Burgers' PDE Near Finite-Time BlowUp

Approximate solution of KdV-Burgers equation using improved PINNs algorithm

Convergence and error analysis of PINNs

A Combination of Deep Neural Networks and Physics to Solve the Inverse Problem of Burger's Equation

Binary structured physics-informed neural networks for solving equations with rapidly changing solutions

Distributed physics informed neural network for data-efficient solution to partial differential equations

Efficient Error Certification for Physics-Informed Neural Networks

Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs

On the estimation rate of Bayesian PINN for inverse problems

Convergence Analysis for the PINNs

Physics Informed Neural Networks (PINNs) as intelligent computing technique for solving partial differential equations: Limitation and Future prospects

Semi-analytic PINN methods for singularly perturbed boundary value problems

General-Kindred Physics-Informed Neural Network to the Solutions of Singularly Perturbed Differential Equations

Burgers' pinns with implicit euler transfer learning

PPINN: Parareal physics-informed neural network for time-dependent PDEs

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs

Enhancing PINNs for solving PDEs via adaptive collocation point movement and adaptive loss weighting

Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the Buckley-Leverett problem

A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations

Improving PINNs By Algebraic Inclusion of Boundary and Initial Conditions