Abstract:We find probability error bounds for approximations of functions $f$ in a separable reproducing kernel Hilbert space $\mathcal{H}$ with reproducing kernel $K$ on a base space $X$, firstly in terms of finite linear combinations of functions of type $K_{x_i}$ and then in terms of the projection $\pi^n_x$ on $\mathrm{Span}\{K_{x_i}\}^n_{i=1}$, for random sequences of points $x=(x_i)_i$ in $X$. Given a probability measure $P$, letting $P_K$ be the measure defined by $\mathrm{d} P_K(x)=K(x,x)\mathrm{d} P(x)$, $x\in X$, our approach is based on the nonexpansive operator \[L^2(X;P_K)\ni\lambda\mapsto L_{P,K}\lambda:=\int_X \lambda(x)K_x\mathrm{d} P(x)\in \mathcal{H},\] where the integral exists in the Bochner sense. Using this operator, we then define a new reproducing kernel Hilbert space, denoted by $\mathcal{H}_P$, that is the operator range of $L_{P,K}$. Our main result establishes bounds, in terms of the operator $L_{P,K}$, on the probability that the Hilbert space distance between an arbitrary function $f\in\mathcal{H}$ and linear combinations of functions of type $K_{x_i}$, for $(x_i)_i$ sampled independently from $P$, falls below a given threshold. For sequences of points $(x_i)_{i=1}^\infty$ constituting a so-called uniqueness set, the orthogonal projections $\pi^n_x$ to $\mathrm{Span}\{K_{x_i}\}^n_{i=1}$ converge in the strong operator topology to the identity operator. We prove that, under the assumption that $\mathcal{H}_P$ is dense in $\mathcal{H}$, any sequence of iid samples from $P$ yields a uniqueness set with probability $1$. This result improves on previous error bounds in weaker norms, such as uniform or $L^p$ norms, which yield only convergence in probability and not a.c. convergence. Two examples that show the applicability of this result to a uniform distribution on a compact interval and to the Hardy space $H^2(\mathbb{D})$ are presented as well.

Approximating Reproducing Kernel Hilbert Space Functions by Bernstein Operators

A new kernel method for the uniform approximation in reproducing kernel Hilbert spaces

On the Approximation of Kernel functions

Approximation properties of certain operator-induced norms on Hilbert spaces

Full length article: Approximation by multivariate Bernstein-Durrmeyer operators and learning rates of least-squares regularized regression with multivariate polynomial kernels

On the Probabilistic Approximation in Reproducing Kernel Hilbert Spaces

Approximation processes by multidimensional Bernstein-type exponential polynomials on the hypercube

Approximation in Hilbert spaces of the Gaussian and related analytic kernels

Bernstein approximation and beyond: proofs by means of elementary probability theory

A sufficient condition for n-Best Kernel Approximation in Reproducing Kernel Hilbert Spaces

Approximation of RKHS Functionals by Neural Networks

Reproducing kernel Banach spaces for machine learning

Probability error bounds for approximation of functions in reproducing kernel Hilbert spaces

Almost-sure quasi-optimal approximation in reproducing kernel Hilbert spaces

Approximate Representer Theorems in Non-reflexive Banach Spaces

Bernstein approximation of optimal control problems

Approximation of the Koopman operator via Bernstein polynomials

Fractional $α$-Bernstein-Kantorovich operators of order $β$: A new construction and approximation results

Generalized Bernstein polynomials

Analysis of Approximation by Linear Operators on VariableLρp(·)Spaces and Applications in Learning Theory

Approximation by bivariate Bernstein–Kantorovich–Stancu operators that reproduce exponential functions