Abstract:SIAM Journal on Scientific Computing, Volume 44, Issue 5, Page A3158-A3182, October 2022. Physics-informed neural networks (PINNs) have become a popular choice for solving high-dimensional partial differential equations (PDEs) due to their excellent approximation power and generalization ability. Recently, extended PINNs (XPINNs) based on domain decomposition methods have attracted considerable attention due to their effectiveness in modeling multiscale and multiphysics problems and their parallelization. However, theoretical understanding of their convergence and generalization properties remains unexplored. In this study, we take an initial step towards understanding how and when XPINNs outperform PINNs. Specifically, for general multilayer PINNs and XPINNs, we first provide a prior generalization bound via the complexity of the target functions in the PDE problem and a posterior generalization bound via the posterior matrix norms of the networks after optimization. Moreover, based on our bounds, we analyze the conditions under which XPINNs improve generalization. Concretely, our theory shows that the key building block of XPINN, namely, the domain decomposition, introduces a tradeoff for generalization. On the one hand, XPINNs decompose the complex PDE solution into several simple parts, which decreases the complexity needed to learn each part and boosts generalization. On the other hand, decomposition leads to less training data being available in each subdomain, and hence such a model is typically prone to overfitting and may become less generalizable. Empirically, we choose five PDEs to show when XPINNs perform better than, similar to, or worse than PINNs, hence demonstrating and justifying our new theory.

Refined generalization analysis of the Deep Ritz Method and Physics-Informed Neural Networks

Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality

Statistical Numerical PDE : Fast Rate, Neural Scaling Law and When it’s Optimal

Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs

Generalization of PINNs for elliptic interface problems

Deep Ritz Methods for Laplace Equations with Dirichlet Boundary Condition

When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?

Residual-based error bound for physics-informed neural networks

Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs

Higher-order error estimates for physics-informed neural networks approximating the primitive equations

Convergence Rate Analysis for Deep Ritz Method

Convergence and error analysis of PINNs

Error Analysis of Deep Ritz Methods for Elliptic Equations.

A Rate of Convergence of Physics Informed Neural Networks for the Linear Second Order Elliptic PDEs

Convergence Analysis for the PINNs

On the Generalization of PINNs outside the training domain and the Hyperparameters influencing it

Advancing Generalization in PINNs through Latent-Space Representations

On the Compatibility between Neural Networks and Partial Differential Equations for Physics-informed Learning

RBF-MGN:Solving spatiotemporal PDEs with Physics-informed Graph Neural Network

A Priori Estimation of the Approximation, Optimization and Generalization Errors of Random Neural Networks for Solving Partial Differential Equations

A Unified Framework for the Error Analysis of Physics-Informed Neural Networks