Abstract:Operator learning, the approximation of mappings between infinite-dimensional function spaces using ideas from machine learning, has gained increasing research attention in recent years. Approximate operators, learned from data, hold promise to serve as efficient surrogate models for problems in computational science and engineering, complementing traditional numerical methods. However, despite their empirical success, our understanding of the underpinning mathematical theory is in large part still incomplete. In this paper, we study the approximation of Lipschitz operators in expectation with respect to Gaussian measures. We prove higher Gaussian Sobolev regularity of Lipschitz operators and establish lower and upper bounds on the Hermite polynomial approximation error. We further consider the reconstruction of Lipschitz operators from $m$ arbitrary (adaptive) linear samples. A key finding is the tight characterization of the smallest achievable error for all possible (adaptive) sampling and reconstruction maps in terms of $m$. It is shown that Hermite polynomial approximation is an optimal recovery strategy, but we have the following curse of sample complexity: No method to approximate Lipschitz operators based on finitely many samples can achieve algebraic convergence rates in $m$. On the positive side, we prove that a sufficiently fast spectral decay of the covariance operator of the Gaussian measure guarantees convergence rates which are arbitrarily close to any algebraic rate in the large data limit $m \to \infty$. Finally, we focus on the recovery of Lipschitz operators from finitely many point samples. We consider Christoffel sampling and weighted least-squares approximation, and present an algorithm which provably achieves near-optimal sample complexity.

Full length article: Approximation by multivariate Bernstein-Durrmeyer operators and learning rates of least-squares regularized regression with multivariate polynomial kernels

Approximation of Sums of Locally Dependent Random Variables via Perturbation of Stein Operator

Learning Rates of Least-Square Regularized Regression with Polynomial Kernels

Analysis of Approximation by Linear Operators on Variable Lρp(·) Spaces and Applications in Learning Theory

Approximation processes by multidimensional Bernstein-type exponential polynomials on the hypercube

On the Approximation of Kernel functions

Learning Theory Viewpoint of Approximation by Positive Linear Operators

Approximating Reproducing Kernel Hilbert Space Functions by Bernstein Operators

Approximation in Hilbert spaces of the Gaussian and related analytic kernels

Error Bounds for a Kernel-Based Constrained Optimal Smoothing Approximation

Analysis of Multivariate Gegenbauer Approximation in the Hypercube

Improved Bounds for the Nyström Method with Application to Kernel Classification

Extending error bounds for radial basis function interpolation to measuring the error in higher order Sobolev norms

High-Dimensional Penalized Bernstein Support Vector Machines

Approximation of discontinuous functions by positive linear operators. A probabilistic approach

Approximation on Variable Exponent Spaces by Linear Integral Operators

Analysis of Least Squares Regularized Regression in Reproducing Kernel Krein Spaces

Kernel Based Approximation in Sobolev Spaces with Radial Basis Functions

Uniform Approximation by Polynomials with Integer Coefficients via the Bernstein Lattice

Learning Lipschitz Operators with respect to Gaussian Measures with Near-Optimal Sample Complexity