Abstract:We establish universality and expression rate bounds for a class of neural Deep Operator Networks (DON) emulating Lipschitz (or Hölder) continuous maps $\mathcal G:\mathcal X\to\mathcal Y$ between (subsets of) separable Hilbert spaces $\mathcal X$, $\mathcal Y$. The DON architecture considered uses linear encoders $\mathcal E$ and decoders $\mathcal D$ via (biorthogonal) Riesz bases of $\mathcal X$, $\mathcal Y$, and an approximator network of an infinite-dimensional, parametric coordinate map that is Lipschitz continuous on the sequence space $\ell^2(\mathbb N)$. Unlike previous works ([Herrmann, Schwab and Zech: Neural and Spectral operator surrogates: construction and expression rate bounds, SAM Report, 2022], [Marcati and Schwab: Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations, SAM Report, 2022]), which required for example $\mathcal G$ to be holomorphic, the present expression rate results require mere Lipschitz (or Hölder) continuity of $\mathcal G$. Key in the proof of the present expression rate bounds is the use of either super-expressive activations (e.g. [Yarotski: Elementary superexpressive activations, Int. Conf. on ML, 2021], [Shen, Yang and Zhang: Neural network approximation: Three hidden layers are enough, Neural Networks, 2021], and the references there) which are inspired by the Kolmogorov superposition theorem, or of nonstandard NN architectures with standard (ReLU) activations as recently proposed in [Zhang, Shen and Yang: Neural Network Architecture Beyond Width and Depth, Adv. in Neural Inf. Proc. Sys., 2022]. We illustrate the abstract results by approximation rate bounds for emulation of a) solution operators for parametric elliptic variational inequalities, and b) Lipschitz maps of Hilbert-Schmidt operators.

Expression Rates of Neural Operators for Linear Elliptic PDEs in Polytopes

Neural and spectral operator surrogates: unified construction and expression rate bounds

Deep Operator Network Approximation Rates for Lipschitz Operators

Quantitative Approximation for Neural Operators in Nonlinear Parabolic Equations

DNN Expression Rate Analysis of High-dimensional PDEs: Application to Option Pricing

Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations

Diffeomorphic Latent Neural Operators for Data-Efficient Learning of Solutions to Partial Differential Equations

Parametric PDEs: Sparse or Low-Rank Approximations?

Exponential Expressivity of ReLU$^k$ Neural Networks on Gevrey Classes with Point Singularities

Approximation of Solution Operators for High-dimensional PDEs

Solving the Poisson Equation with Dirichlet data by shallow ReLU$^α$-networks: A regularity and approximation perspective

Neural Operator: Learning Maps Between Function Spaces

Neural Operator: Graph Kernel Network for Partial Differential Equations

Operator learning for hyperbolic partial differential equations

Optimal Convergence Rates for Neural Operators

Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality

Operator Learning of Lipschitz Operators: An Information-Theoretic Perspective

Neural Control of Parametric Solutions for High-dimensional Evolution PDEs

Nonlinear Reconstruction for Operator Learning of PDEs with Discontinuities

Neural Green's Operators for Parametric Partial Differential Equations

Learning Partial Differential Equations with Deep Parallel Neural Operator