Abstract:Operator learning, the approximation of mappings between infinite-dimensional function spaces using ideas from machine learning, has gained increasing research attention in recent years. Approximate operators, learned from data, hold promise to serve as efficient surrogate models for problems in computational science and engineering, complementing traditional numerical methods. However, despite their empirical success, our understanding of the underpinning mathematical theory is in large part still incomplete. In this paper, we study the approximation of Lipschitz operators in expectation with respect to Gaussian measures. We prove higher Gaussian Sobolev regularity of Lipschitz operators and establish lower and upper bounds on the Hermite polynomial approximation error. We further consider the reconstruction of Lipschitz operators from $m$ arbitrary (adaptive) linear samples. A key finding is the tight characterization of the smallest achievable error for all possible (adaptive) sampling and reconstruction maps in terms of $m$. It is shown that Hermite polynomial approximation is an optimal recovery strategy, but we have the following curse of sample complexity: No method to approximate Lipschitz operators based on finitely many samples can achieve algebraic convergence rates in $m$. On the positive side, we prove that a sufficiently fast spectral decay of the covariance operator of the Gaussian measure guarantees convergence rates which are arbitrarily close to any algebraic rate in the large data limit $m \to \infty$. Finally, we focus on the recovery of Lipschitz operators from finitely many point samples. We consider Christoffel sampling and weighted least-squares approximation, and present an algorithm which provably achieves near-optimal sample complexity.

Learning optimal smooth invariant subspaces for data approximation

Infinite-Variate $L^2$-Approximation with Nested Subspace Sampling

On the approximation of vector-valued functions by volume sampling

Smooth Distance Approximation

Learning Lipschitz Operators with respect to Gaussian Measures with Near-Optimal Sample Complexity

Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks

Smoothed Analysis for Learning Concepts with Low Intrinsic Dimension

Building Invariances into Sparse Subspace Clustering

Optimal approximation of infinite-dimensional holomorphic functions

Learning Smooth Distance Functions via Queries

Subsampling in Smoothed Range Spaces

Operator learning based on sparse high-dimensional approximation

Multivariate approximation of functions on irregular domains by weighted least-squares methods

A Subspace Learning Method Based on Polynomial Smoothing

APPROXIMATING SMOOTH, MULTIVARIATE FUNCTIONS ON IRREGULAR DOMAINS

Constructing Approximations to Bivariate Piecewise-Smooth Functions

Approximation in Hölder Spaces

Optimal low-rank approximations of posteriors for linear Gaussian inverse problems on Hilbert spaces

Almost-sure quasi-optimal approximation in reproducing kernel Hilbert spaces

Stein's method, smoothing and functional approximation