Convergence and error analysis of PINNs

Nathan Doumèche,Gérard Biau,Claire Boyer

2023-05-02

Abstract:Physics-informed neural networks (PINNs) are a promising approach that combines the power of neural networks with the interpretability of physical modeling. PINNs have shown good practical performance in solving partial differential equations (PDEs) and in hybrid modeling scenarios, where physical models enhance data-driven approaches. However, it is essential to establish their theoretical properties in order to fully understand their capabilities and limitations. In this study, we highlight that classical training of PINNs can suffer from systematic overfitting. This problem can be addressed by adding a ridge regularization to the empirical risk, which ensures that the resulting estimator is risk-consistent for both linear and nonlinear PDE systems. However, the strong convergence of PINNs to a solution satisfying the physical constraints requires a more involved analysis using tools from functional analysis and calculus of variations. In particular, for linear PDE systems, an implementable Sobolev-type regularization allows to reconstruct a solution that not only achieves statistical accuracy but also maintains consistency with the underlying physics.

Statistics Theory

What problem does this paper attempt to address?

The paper primarily focuses on the theoretical properties and performance issues of Physics-Informed Neural Networks (PINNs) in solving Partial Differential Equations (PDEs) and hybrid modeling scenarios.

Specifically, the paper attempts to address the following core issues:

1. **Systematic Overfitting Problem**: Traditional PINNs training processes may encounter systematic overfitting issues. This means that although PINNs perform well in practical applications, they lack sufficient theoretical guarantees to avoid overfitting.

2. **Insufficient Theoretical Analysis**: Despite the good performance of PINNs in various practical application scenarios, there is still relatively little theoretical analysis of PINNs, especially regarding overfitting, consistency, and error analysis.

3. **Improving Training Methods to Ensure Consistency**: To overcome these issues, the paper proposes an improved training method—by adding a ridge regularization term to the empirical risk, it can ensure the risk consistency of the estimator for both linear and nonlinear PDE systems.

4. **Strong Convergence Analysis**: Furthermore, to ensure that PINNs can strongly converge to solutions that satisfy physical constraints, the paper provides a more in-depth analysis using tools from functional analysis and variational methods.

In summary, this paper aims to provide a more solid theoretical foundation for PINNs through theoretical analysis and mathematical proofs, and proposes practical methods to improve the performance of PINNs, particularly in terms of consistency and convergence when solving PDEs and hybrid modeling problems.