Abstract:Recently, several deep learning (DL) methods for approximating high-dimensional partial differential equations (PDEs) have been proposed. The interest that these methods have generated in the literature is in large part due to simulations which appear to demonstrate that such DL methods have the capacity to overcome the curse of dimensionality (COD) for PDEs in the sense that the number of computational operations they require to achieve a certain approximation accuracy $\varepsilon\in(0,\infty)$ grows at most polynomially in the PDE dimension $d\in\mathbb N$ and the reciprocal of $\varepsilon$. While there is thus far no mathematical result that proves that one of such methods is indeed capable of overcoming the COD, there are now a number of rigorous results in the literature that show that deep neural networks (DNNs) have the expressive power to approximate PDE solutions without the COD in the sense that the number of parameters used to describe the approximating DNN grows at most polynomially in both the PDE dimension $d\in\mathbb N$ and the reciprocal of the approximation accuracy $\varepsilon>0$. Roughly speaking, in the literature it is has been proved for every $T>0$ that solutions $u_d\colon [0,T]\times\mathbb R^d\to \mathbb R$, $d\in\mathbb N$, of semilinear heat PDEs with Lipschitz continuous nonlinearities can be approximated by DNNs with ReLU activation at the terminal time in the $L^2$-sense without the COD provided that the initial value functions $\mathbb R^d\ni x\mapsto u_d(0,x)\in\mathbb R$, $d\in\mathbb N$, can be approximated by ReLU DNNs without the COD. It is the key contribution of this work to generalize this result by establishing this statement in the $L^p$-sense with $p\in(0,\infty)$ and by allowing the activation function to be more general covering the ReLU, the leaky ReLU, and the softplus activation functions as special cases.

Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theorem

Approximation Error and Complexity Bounds for ReLU Networks on Low-Regular Function Spaces

The Kolmogorov Superposition Theorem can Break the Curse of Dimensionality When Approximating High Dimensional Functions

Error bounds for approximations with deep ReLU neural networks in $W^{s,p}$ norms

The Kolmogorov-Arnold representation theorem revisited

Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks

Neural networks with ReLU powers need less depth

Nearly Optimal Approximation Rates for Deep Super ReLU Networks on Sobolev Spaces

Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces

Towards Lower Bounds on the Depth of ReLU Neural Networks

On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks

Three Quantization Regimes for ReLU Networks

Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on Hölder Class.

Implicit Hypersurface Approximation Capacity in Deep ReLU Networks

Approximation in Shift-Invariant Spaces with Deep ReLU Neural Networks

Approximation Rates for Shallow ReLU$^k$ Neural Networks on Sobolev Spaces via the Radon Transform

Universal Consistency of Wide and Deep ReLU Neural Networks and Minimax Optimal Convergence Rates for Kolmogorov-Donoho Optimal Function Classes

Approximation of functions from Korobov spaces by shallow neural networks

Depth Separations in Neural Networks: Separating the Dimension from the Accuracy

Lower bounds for artificial neural network approximations: A proof that shallow neural networks fail to overcome the curse of dimensionality

Deep neural networks with ReLU, leaky ReLU, and softplus activation provably overcome the curse of dimensionality for Kolmogorov partial differential equations with Lipschitz nonlinearities in the $L^p$-sense