Abstract:The past decade has seen increasing interest in applying Deep Learning (DL) to Computational Science and Engineering (CSE). Driven by impressive results in applications such as computer vision, Uncertainty Quantification (UQ), genetics, simulations and image processing, DL is increasingly supplanting classical algorithms, and seems poised to revolutionize scientific computing. However, DL is not yet well-understood from the standpoint of numerical analysis. Little is known about the efficiency and reliability of DL from the perspectives of stability, robustness, accuracy, and sample complexity. In particular, approximating solutions to parametric PDEs is an objective of UQ for CSE. Training data for such problems is often scarce and corrupted by errors. Moreover, the target function is a possibly infinite-dimensional smooth function taking values in the PDE solution space, generally an infinite-dimensional Banach space. This paper provides arguments for Deep Neural Network (DNN) approximation of such functions, with both known and unknown parametric dependence, that overcome the curse of dimensionality. We establish practical existence theorems that describe classes of DNNs with dimension-independent architecture size and training procedures based on minimizing the (regularized) $\ell^2$-loss which achieve near-optimal algebraic rates of convergence. These results involve key extensions of compressed sensing for Banach-valued recovery and polynomial emulation with DNNs. When approximating solutions of parametric PDEs, our results account for all sources of error, i.e., sampling, optimization, approximation and physical discretization, and allow for training high-fidelity DNN approximations from coarse-grained sample data. Our theoretical results fall into the category of non-intrusive methods, providing a theoretical alternative to classical methods for high-dimensional approximation.

Deep neural network expressivity for optimal stopping problems

A New Deep Neural Network Algorithm for Multiple Stopping with Applications in Options Pricing

DNN Expression Rate Analysis of High-dimensional PDEs: Application to Option Pricing

Optimal Stopping via Randomized Neural Networks

A deep primal-dual BSDE method for optimal stopping problems

Nonparametric regression using deep neural networks with ReLU activation function

Neural networks with ReLU powers need less depth

Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on Hölder Class.

Neural optimal stopping boundary

On the Expressive Power of Neural Networks

A deep learning method for pricing high-dimensional American-style options via state-space partition

Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces

Deep Learning Algorithms for Mean Field Optimal Stopping in Finite Space and Discrete Time

Optimal rates of approximation by shallow ReLU$^k$ neural networks and applications to nonparametric regression

Approximation Rates for Deep Calibration of (Rough) Stochastic Volatility Models

Generalization and Expressivity for Deep Nets

Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality

Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks

Solving the optimal stopping problem with reinforcement learning: an application in financial option exercise

Optimal Stopping for Strong Markov Processes: Explicit solutions and verification theorems for diffusions, multidimensional diffusions, and jump-processes

Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks