Abstract:The past decade has seen increasing interest in applying Deep Learning (DL) to Computational Science and Engineering (CSE). Driven by impressive results in applications such as computer vision, Uncertainty Quantification (UQ), genetics, simulations and image processing, DL is increasingly supplanting classical algorithms, and seems poised to revolutionize scientific computing. However, DL is not yet well-understood from the standpoint of numerical analysis. Little is known about the efficiency and reliability of DL from the perspectives of stability, robustness, accuracy, and sample complexity. In particular, approximating solutions to parametric PDEs is an objective of UQ for CSE. Training data for such problems is often scarce and corrupted by errors. Moreover, the target function is a possibly infinite-dimensional smooth function taking values in the PDE solution space, generally an infinite-dimensional Banach space. This paper provides arguments for Deep Neural Network (DNN) approximation of such functions, with both known and unknown parametric dependence, that overcome the curse of dimensionality. We establish practical existence theorems that describe classes of DNNs with dimension-independent architecture size and training procedures based on minimizing the (regularized) $\ell^2$-loss which achieve near-optimal algebraic rates of convergence. These results involve key extensions of compressed sensing for Banach-valued recovery and polynomial emulation with DNNs. When approximating solutions of parametric PDEs, our results account for all sources of error, i.e., sampling, optimization, approximation and physical discretization, and allow for training high-fidelity DNN approximations from coarse-grained sample data. Our theoretical results fall into the category of non-intrusive methods, providing a theoretical alternative to classical methods for high-dimensional approximation.

On the learning of high order polynomial reconstructions for essentially non-oscillatory schemes

Reconstruction of Noise-Driven Nonlinear Networks from Node Outputs by Using High-Order Correlations

NEURAL ORDINARY DIFFERENTIAL EQUATIONS FOR TIME SERIES RECONSTRUCTION

WENO reconstructions of unconditionally optimal high order

A third-order finite difference weighted essentially non-oscillatory scheme with shallow neural network

Learning WENO for entropy stable schemes to solve conservation laws

Exponential approximation space reconstruction WENO scheme for dispersive PDEs

Nonlinear Reconstruction for Operator Learning of PDEs with Discontinuities

On the approximation of rough functions with deep neural networks

An efficient third-order WENO scheme with unconditionally optimal accuracy

Precision Learning: Reconstruction Filter Kernel Discretization

Generalization Error Bounds for Iterative Recovery Algorithms Unfolded as Neural Networks

Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses

Rational-WENO: A lightweight, physically-consistent three-point weighted essentially non-oscillatory scheme

ReLUs Are Sufficient for Learning Implicit Neural Representations

Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks

Anamnesic Neural Differential Equations with Orthogonal Polynomial Projections

Learning Fast Approximations of Sparse Nonlinear Regression

Solving Non-linear Kolmogorov Equations in Large Dimensions by Using Deep Learning: A Numerical Comparison of Discretization Schemes

Deep synthesis regularization of inverse problems

Deep-learning reconstruction of complex dynamical networks from incomplete data