Abstract:We establish universality and expression rate bounds for a class of neural Deep Operator Networks (DON) emulating Lipschitz (or Hölder) continuous maps $\mathcal G:\mathcal X\to\mathcal Y$ between (subsets of) separable Hilbert spaces $\mathcal X$, $\mathcal Y$. The DON architecture considered uses linear encoders $\mathcal E$ and decoders $\mathcal D$ via (biorthogonal) Riesz bases of $\mathcal X$, $\mathcal Y$, and an approximator network of an infinite-dimensional, parametric coordinate map that is Lipschitz continuous on the sequence space $\ell^2(\mathbb N)$. Unlike previous works ([Herrmann, Schwab and Zech: Neural and Spectral operator surrogates: construction and expression rate bounds, SAM Report, 2022], [Marcati and Schwab: Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations, SAM Report, 2022]), which required for example $\mathcal G$ to be holomorphic, the present expression rate results require mere Lipschitz (or Hölder) continuity of $\mathcal G$. Key in the proof of the present expression rate bounds is the use of either super-expressive activations (e.g. [Yarotski: Elementary superexpressive activations, Int. Conf. on ML, 2021], [Shen, Yang and Zhang: Neural network approximation: Three hidden layers are enough, Neural Networks, 2021], and the references there) which are inspired by the Kolmogorov superposition theorem, or of nonstandard NN architectures with standard (ReLU) activations as recently proposed in [Zhang, Shen and Yang: Neural Network Architecture Beyond Width and Depth, Adv. in Neural Inf. Proc. Sys., 2022]. We illustrate the abstract results by approximation rate bounds for emulation of a) solution operators for parametric elliptic variational inequalities, and b) Lipschitz maps of Hilbert-Schmidt operators.

$L^{p}$-convergence of Kantorovich-type Max-Min Neural Network Operators

Exponential Sampling Type Kantorovich Max-Product Neural Network Operators

Approximation Results in Orlicz Spaces for Sequences of Kantorovich Max-Product Neural Network Operators

Best Approximation and Inverse Results for Neural Network Operators

Kantorovich-Type Convergence Criterion for Inexact Newton Methods

Max-Product Sampling Kantorovich Operators: Quantitative Estimates in Functional Spaces

A characterization of the rate of the simultaneous approximation by generalized sampling operators and their Kantorovich modification

Max-product Kantorovich sampling operators: quantitative estimates in functional spaces

Approximation properties and q-statistical convergence of Kantorovich variant of Stancu type Lupaş operators

Approximation by Kantorovich Type q-Bernstein-Stancu Operators

Certain Approximation Results for Kantorovich Exponential Sampling Series

Optimal Convergence Rates for Neural Operators

Direct Estimates of the Rate of Approximation by the Kantorovich Operator in Variable Exponent Lebesgue Spaces

Asymptotic Analysis of Neural Network Operators Employing the Hardy-Littlewood Maximal Inequality

Operator Learning of Lipschitz Operators: An Information-Theoretic Perspective

Approximation by bivariate Bernstein–Kantorovich–Stancu operators that reproduce exponential functions

MinMax Networks

Ergodic properties of Kantorovich operators

Approximating Iterations for Nonexpansive and Maximal Monotone Operators

Deep Operator Network Approximation Rates for Lipschitz Operators

Convergence analysis of wide shallow neural operators within the framework of Neural Tangent Kernel