Abstract:We find probability error bounds for approximations of functions $f$ in a separable reproducing kernel Hilbert space $\mathcal{H}$ with reproducing kernel $K$ on a base space $X$, firstly in terms of finite linear combinations of functions of type $K_{x_i}$ and then in terms of the projection $\pi^n_x$ on $\mathrm{Span}\{K_{x_i}\}^n_{i=1}$, for random sequences of points $x=(x_i)_i$ in $X$. Given a probability measure $P$, letting $P_K$ be the measure defined by $\mathrm{d} P_K(x)=K(x,x)\mathrm{d} P(x)$, $x\in X$, our approach is based on the nonexpansive operator \[L^2(X;P_K)\ni\lambda\mapsto L_{P,K}\lambda:=\int_X \lambda(x)K_x\mathrm{d} P(x)\in \mathcal{H},\] where the integral exists in the Bochner sense. Using this operator, we then define a new reproducing kernel Hilbert space, denoted by $\mathcal{H}_P$, that is the operator range of $L_{P,K}$. Our main result establishes bounds, in terms of the operator $L_{P,K}$, on the probability that the Hilbert space distance between an arbitrary function $f\in\mathcal{H}$ and linear combinations of functions of type $K_{x_i}$, for $(x_i)_i$ sampled independently from $P$, falls below a given threshold. For sequences of points $(x_i)_{i=1}^\infty$ constituting a so-called uniqueness set, the orthogonal projections $\pi^n_x$ to $\mathrm{Span}\{K_{x_i}\}^n_{i=1}$ converge in the strong operator topology to the identity operator. We prove that, under the assumption that $\mathcal{H}_P$ is dense in $\mathcal{H}$, any sequence of iid samples from $P$ yields a uniqueness set with probability $1$. This result improves on previous error bounds in weaker norms, such as uniform or $L^p$ norms, which yield only convergence in probability and not a.c. convergence. Two examples that show the applicability of this result to a uniform distribution on a compact interval and to the Hardy space $H^2(\mathbb{D})$ are presented as well.

Which Spaces can be Embedded in $L_p$-type Reproducing Kernel Banach Space? A Characterization via Metric Entropy

The $L^\infty$ Learnability of Reproducing Kernel Hilbert Spaces

On embedding separable spaces $\mathcal{C}(L)$ in arbitrary spaces $\mathcal{C}(K)$

Reproducing kernels and choices of associated feature spaces, in the form of $L^{2}$-spaces

A reproducing kernel approach to Lebesgue decomposition

Characterization of LIL Behavior for Non-Degenerate B-Valued U-Statistics

Probability error bounds for approximation of functions in reproducing kernel Hilbert spaces

Kernel embedding of measures and low-rank approximation of integral operators

Canonical embedding of Lipschitz-free $p$-spaces

Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spaces

Approximate Representer Theorems in Non-reflexive Banach Spaces

Entropy numbers of finite-dimensional Lorentz space embeddings

Learning to Embed Distributions via Maximum Kernel Entropy

On the Metric Distortion of Embedding Persistence Diagrams into separable Hilbert spaces

Entropy numbers of Reproducing Hilbert Space of zonal positive definite kernels on compact two-point homogeneous spaces

Markov Type and Threshold Embeddings

Nonuniform sampling, reproducing kernels, and the associated Hilbert spaces

New characterizations of reproducing kernel Hilbert spaces and applications to metric geometry

Discrete logarithmic Sobolev inequalities in Banach spaces

Neural reproducing kernel Banach spaces and representer theorems for deep networks

Adaptive indefinite kernels in hyperbolic spaces