Abstract:The past decade has seen increasing interest in applying Deep Learning (DL) to Computational Science and Engineering (CSE). Driven by impressive results in applications such as computer vision, Uncertainty Quantification (UQ), genetics, simulations and image processing, DL is increasingly supplanting classical algorithms, and seems poised to revolutionize scientific computing. However, DL is not yet well-understood from the standpoint of numerical analysis. Little is known about the efficiency and reliability of DL from the perspectives of stability, robustness, accuracy, and sample complexity. In particular, approximating solutions to parametric PDEs is an objective of UQ for CSE. Training data for such problems is often scarce and corrupted by errors. Moreover, the target function is a possibly infinite-dimensional smooth function taking values in the PDE solution space, generally an infinite-dimensional Banach space. This paper provides arguments for Deep Neural Network (DNN) approximation of such functions, with both known and unknown parametric dependence, that overcome the curse of dimensionality. We establish practical existence theorems that describe classes of DNNs with dimension-independent architecture size and training procedures based on minimizing the (regularized) $\ell^2$-loss which achieve near-optimal algebraic rates of convergence. These results involve key extensions of compressed sensing for Banach-valued recovery and polynomial emulation with DNNs. When approximating solutions of parametric PDEs, our results account for all sources of error, i.e., sampling, optimization, approximation and physical discretization, and allow for training high-fidelity DNN approximations from coarse-grained sample data. Our theoretical results fall into the category of non-intrusive methods, providing a theoretical alternative to classical methods for high-dimensional approximation.

Deep Neural Network Approximation of Composition Functions: with Application to PINNs

Improved Analysis of PINNs: Alleviate the CoD for Compositional Solutions

Neural Network Approximations of Compositional Functions With Applications to Dynamical Systems

Deep neural network approximation of composite functions without the curse of dimensionality

Error Analysis and Numerical Algorithm for PDE Approximation with Hidden-Layer Concatenated Physics Informed Neural Networks

On the Approximation and Complexity of Deep Neural Networks to Invariant Functions

Why Deep Neural Networks for Function Approximation?

Deep Nonparametric Regression on Approximate Manifolds: Non-Asymptotic Error Bounds with Polynomial Prefactors

Error Analysis of Physics-Informed Neural Networks for Approximating Dynamic PDEs of Second Order in Time

Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks

Approximation smooth and sparse functions by deep neural networks without saturation

Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs

Deep neural network approximations for Monte Carlo algorithms

wPINNs: Weak Physics Informed Neural Networks for Approximating Entropy Solutions of Hyperbolic Conservation Laws

Compositional Sparsity, Approximation Classes, and Parametric Transport Equations

Convergence Analysis for the PINNs

Approximations of Continuous Functionals by Neural Networks with Application to Dynamic Systems

Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs

Functional Tensor Decompositions for Physics-Informed Neural Networks

Invariant deep neural networks under the finite group for solving partial differential equations

VI-PINNs: Variance-involved Physics-informed Neural Networks for Fast and Accurate Prediction of Partial Differential Equations