Abstract:We establish universality and expression rate bounds for a class of neural Deep Operator Networks (DON) emulating Lipschitz (or Hölder) continuous maps $\mathcal G:\mathcal X\to\mathcal Y$ between (subsets of) separable Hilbert spaces $\mathcal X$, $\mathcal Y$. The DON architecture considered uses linear encoders $\mathcal E$ and decoders $\mathcal D$ via (biorthogonal) Riesz bases of $\mathcal X$, $\mathcal Y$, and an approximator network of an infinite-dimensional, parametric coordinate map that is Lipschitz continuous on the sequence space $\ell^2(\mathbb N)$. Unlike previous works ([Herrmann, Schwab and Zech: Neural and Spectral operator surrogates: construction and expression rate bounds, SAM Report, 2022], [Marcati and Schwab: Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations, SAM Report, 2022]), which required for example $\mathcal G$ to be holomorphic, the present expression rate results require mere Lipschitz (or Hölder) continuity of $\mathcal G$. Key in the proof of the present expression rate bounds is the use of either super-expressive activations (e.g. [Yarotski: Elementary superexpressive activations, Int. Conf. on ML, 2021], [Shen, Yang and Zhang: Neural network approximation: Three hidden layers are enough, Neural Networks, 2021], and the references there) which are inspired by the Kolmogorov superposition theorem, or of nonstandard NN architectures with standard (ReLU) activations as recently proposed in [Zhang, Shen and Yang: Neural Network Architecture Beyond Width and Depth, Adv. in Neural Inf. Proc. Sys., 2022]. We illustrate the abstract results by approximation rate bounds for emulation of a) solution operators for parametric elliptic variational inequalities, and b) Lipschitz maps of Hilbert-Schmidt operators.

Best Approximation and Inverse Results for Neural Network Operators

Simultaneous approximation by neural network operators with applications to Voronovskaja formulas

The approximation capabilities of Durrmeyer-type neural network operators

Approximation Results in Orlicz Spaces for Sequences of Kantorovich Max-Product Neural Network Operators

Approximation by Exponential Type Neural Network Operators

Deep Operator Network Approximation Rates for Lipschitz Operators

Operator Learning of Lipschitz Operators: An Information-Theoretic Perspective

Kantorovich-Type Convergence Criterion for Inexact Newton Methods

Quantitative Approximation for Neural Operators in Nonlinear Parabolic Equations

On the approximation of bi-Lipschitz maps by invertible neural networks

$L^{p}$-convergence of Kantorovich-type Max-Min Neural Network Operators

Approximation of neural network operators based on the Joukowski transformation

Neural Operator: Learning Maps Between Function Spaces

Asymptotic Analysis of Neural Network Operators Employing the Hardy-Littlewood Maximal Inequality

Approximation by Steklov Neural Network Operators

The New Approximation Operators with Sigmoidal Functions

Operator Learning: Algorithms and Analysis

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

Optimal Convergence Rates for Neural Operators

Approximation by α-Bernstein-Kantorovich Type Operators for Functions of One and Two Variables