Abstract:The past decade has seen increasing interest in applying Deep Learning (DL) to Computational Science and Engineering (CSE). Driven by impressive results in applications such as computer vision, Uncertainty Quantification (UQ), genetics, simulations and image processing, DL is increasingly supplanting classical algorithms, and seems poised to revolutionize scientific computing. However, DL is not yet well-understood from the standpoint of numerical analysis. Little is known about the efficiency and reliability of DL from the perspectives of stability, robustness, accuracy, and sample complexity. In particular, approximating solutions to parametric PDEs is an objective of UQ for CSE. Training data for such problems is often scarce and corrupted by errors. Moreover, the target function is a possibly infinite-dimensional smooth function taking values in the PDE solution space, generally an infinite-dimensional Banach space. This paper provides arguments for Deep Neural Network (DNN) approximation of such functions, with both known and unknown parametric dependence, that overcome the curse of dimensionality. We establish practical existence theorems that describe classes of DNNs with dimension-independent architecture size and training procedures based on minimizing the (regularized) $\ell^2$-loss which achieve near-optimal algebraic rates of convergence. These results involve key extensions of compressed sensing for Banach-valued recovery and polynomial emulation with DNNs. When approximating solutions of parametric PDEs, our results account for all sources of error, i.e., sampling, optimization, approximation and physical discretization, and allow for training high-fidelity DNN approximations from coarse-grained sample data. Our theoretical results fall into the category of non-intrusive methods, providing a theoretical alternative to classical methods for high-dimensional approximation.

A Note on Dimensionality Reduction in Deep Neural Networks using Empirical Interpolation Method

Neural Network Emulation of Spatio-temporal Data Using Linear and Nonlinear Dimensionality Reduction

Nonlinear Model Reduction via Discrete Empirical Interpolation

Space-time deep neural network approximations for high-dimensional partial differential equations

Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks

Neural empirical interpolation method for nonlinear model reduction

Tensor-based empirical interpolation method,\newline and its application in model reduction

An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensions

High-order empirical interpolation methods for real time solution of parametrized nonlinear PDEs

Numerical Solution of the Parametric Diffusion Equation by Deep Neural Networks

Deep Neural Networks for Nonparametric Interaction Models with Diverging Dimension

Efficient and accurate nonlinear model reduction via first-order empirical interpolation

Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations

Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories

Sparsifying dimensionality reduction of PDE solution data with Bregman learning

First-order empirical interpolation method for real-time solution of parametric time-dependent nonlinear PDEs

Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equations

Adaptive Finite Element Interpolated Neural Networks

A hybrid FEM-PINN method for time-dependent partial differential equations

Solving High-Dimensional Partial Differential Equations Using Deep Learning.

Deep neural network approximations for Monte Carlo algorithms