Abstract:We establish universality and expression rate bounds for a class of neural Deep Operator Networks (DON) emulating Lipschitz (or Hölder) continuous maps $\mathcal G:\mathcal X\to\mathcal Y$ between (subsets of) separable Hilbert spaces $\mathcal X$, $\mathcal Y$. The DON architecture considered uses linear encoders $\mathcal E$ and decoders $\mathcal D$ via (biorthogonal) Riesz bases of $\mathcal X$, $\mathcal Y$, and an approximator network of an infinite-dimensional, parametric coordinate map that is Lipschitz continuous on the sequence space $\ell^2(\mathbb N)$. Unlike previous works ([Herrmann, Schwab and Zech: Neural and Spectral operator surrogates: construction and expression rate bounds, SAM Report, 2022], [Marcati and Schwab: Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations, SAM Report, 2022]), which required for example $\mathcal G$ to be holomorphic, the present expression rate results require mere Lipschitz (or Hölder) continuity of $\mathcal G$. Key in the proof of the present expression rate bounds is the use of either super-expressive activations (e.g. [Yarotski: Elementary superexpressive activations, Int. Conf. on ML, 2021], [Shen, Yang and Zhang: Neural network approximation: Three hidden layers are enough, Neural Networks, 2021], and the references there) which are inspired by the Kolmogorov superposition theorem, or of nonstandard NN architectures with standard (ReLU) activations as recently proposed in [Zhang, Shen and Yang: Neural Network Architecture Beyond Width and Depth, Adv. in Neural Inf. Proc. Sys., 2022]. We illustrate the abstract results by approximation rate bounds for emulation of a) solution operators for parametric elliptic variational inequalities, and b) Lipschitz maps of Hilbert-Schmidt operators.

Exponential Sampling Type Kantorovich Max-Product Neural Network Operators

$L^{p}$-convergence of Kantorovich-type Max-Min Neural Network Operators

Certain Approximation Results for Kantorovich Exponential Sampling Series

Approximation by Exponential Type Neural Network Operators

Max-Product Sampling Kantorovich Operators: Quantitative Estimates in Functional Spaces

Max-product Kantorovich sampling operators: quantitative estimates in functional spaces

Direct Estimates of the Rate of Approximation by the Kantorovich Operator in Variable Exponent Lebesgue Spaces

Advancements in nonlinear exponential sampling: convergence, quantitative analysis and Voronovskaya-type formula

A characterization of the rate of the simultaneous approximation by generalized sampling operators and their Kantorovich modification

Weighted Approximation By Max-Product Generalized Exponential Sampling Series

Best Approximation and Inverse Results for Neural Network Operators

Approximation by bivariate Bernstein–Kantorovich–Stancu operators that reproduce exponential functions

Approximation Properties of Mellin-Steklov Type Exponential Sampling Series

Approximation by Kantorovich Type q-Bernstein-Stancu Operators

Approximation of operators on real line

Deep Operator Network Approximation Rates for Lipschitz Operators

Operator Learning of Lipschitz Operators: An Information-Theoretic Perspective

Simultaneous approximation by neural network operators with applications to Voronovskaja formulas

Approximation by a new kind of -Bernstein–Kantorovich operators

Statistical approximation properties of (p, q)-Szasz-Mirakjan Kantorovich operators

Approximation by Steklov Neural Network Operators