Abstract:It is a challenging topic in applied mathematics to solve high-dimensional nonlinear partial differential equations (PDEs). Standard approximation methods for nonlinear PDEs suffer under the curse of dimensionality (COD) in the sense that the number of computational operations of the approximation method grows at least exponentially in the PDE dimension and with such methods it is essentially impossible to approximately solve high-dimensional PDEs even when the fastest currently available computers are used. However, in the last years great progress has been made in this area of research through suitable deep learning (DL) based methods for PDEs in which deep neural networks (DNNs) are used to approximate solutions of PDEs. Despite the remarkable success of such DL methods in simulations, it remains a fundamental open problem of research to prove (or disprove) that such methods can overcome the COD in the approximation of PDEs. However, there are nowadays several partial error analysis results for DL methods for high-dimensional nonlinear PDEs in the literature which prove that DNNs can overcome the COD in the sense that the number of parameters of the approximating DNN grows at most polynomially in both the reciprocal of the prescribed approximation accuracy $\varepsilon>0$ and the PDE dimension $d\in\mathbb{N}$. In the main result of this article we prove that for all $T,p\in(0,\infty)$ it holds that solutions $u_d\colon[0,T]\times\mathbb{R}^d\to\mathbb{R}$, $d\in\mathbb{N}$, of semilinear heat equations with Lipschitz continuous nonlinearities can be approximated in the $L^p$-sense on space-time regions without the COD by DNNs with the rectified linear unit (ReLU), the leaky ReLU, or the softplus activation function. In previous articles similar results have been established not for space-time regions but for the solutions $u_d(T,\cdot)$, $d\in\mathbb{N}$, at the terminal time $T$.

Nearly Optimal Approximation Rates for Deep Super ReLU Networks on Sobolev Spaces

On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks

Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces

Approximation Rates for Shallow ReLU$^k$ Neural Networks on Sobolev Spaces via the Radon Transform

Optimal Rates of Approximation by Shallow ReLU Neural Networks and Applications to Nonparametric Regression

Rates of Approximation by ReLU Shallow Neural Networks

Optimal rates of approximation by shallow ReLU$^k$ neural networks and applications to nonparametric regression

Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on Hölder Class.

Error bounds for approximations with deep ReLU neural networks in $W^{s,p}$ norms

Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks

Approximation in Shift-Invariant Spaces with Deep ReLU Neural Networks

Nonparametric regression using over-parameterized shallow ReLU neural networks

Optimal function approximation with ReLU neural networks

Neural networks with ReLU powers need less depth

Implicit Hypersurface Approximation Capacity in Deep ReLU Networks

Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Norms

Approximation of functions from Korobov spaces by shallow neural networks

Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality

Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for space-time solutions of semilinear partial differential equations

Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces

Deep vs. shallow networks : An approximation theory perspective