Abstract:We find probability error bounds for approximations of functions $f$ in a separable reproducing kernel Hilbert space $\mathcal{H}$ with reproducing kernel $K$ on a base space $X$, firstly in terms of finite linear combinations of functions of type $K_{x_i}$ and then in terms of the projection $\pi^n_x$ on $\mathrm{Span}\{K_{x_i}\}^n_{i=1}$, for random sequences of points $x=(x_i)_i$ in $X$. Given a probability measure $P$, letting $P_K$ be the measure defined by $\mathrm{d} P_K(x)=K(x,x)\mathrm{d} P(x)$, $x\in X$, our approach is based on the nonexpansive operator \[L^2(X;P_K)\ni\lambda\mapsto L_{P,K}\lambda:=\int_X \lambda(x)K_x\mathrm{d} P(x)\in \mathcal{H},\] where the integral exists in the Bochner sense. Using this operator, we then define a new reproducing kernel Hilbert space, denoted by $\mathcal{H}_P$, that is the operator range of $L_{P,K}$. Our main result establishes bounds, in terms of the operator $L_{P,K}$, on the probability that the Hilbert space distance between an arbitrary function $f\in\mathcal{H}$ and linear combinations of functions of type $K_{x_i}$, for $(x_i)_i$ sampled independently from $P$, falls below a given threshold. For sequences of points $(x_i)_{i=1}^\infty$ constituting a so-called uniqueness set, the orthogonal projections $\pi^n_x$ to $\mathrm{Span}\{K_{x_i}\}^n_{i=1}$ converge in the strong operator topology to the identity operator. We prove that, under the assumption that $\mathcal{H}_P$ is dense in $\mathcal{H}$, any sequence of iid samples from $P$ yields a uniqueness set with probability $1$. This result improves on previous error bounds in weaker norms, such as uniform or $L^p$ norms, which yield only convergence in probability and not a.c. convergence. Two examples that show the applicability of this result to a uniform distribution on a compact interval and to the Hardy space $H^2(\mathbb{D})$ are presented as well.

Probabilistic Distance Measures in Reproducing Kernel Hibert Space

Distance-Based Classifier Via the Kernel Trick.

A Parzen-based distance between probability measures as an alternative of summary statistics in Approximate Bayesian Computation

Measures of Entropy From Data Using Infinitely Divisible Kernels

Kernel distance measures for time series, random fields and other structured data

Sliced Wasserstein Kernels for Probability Distributions

Diffeomorphic Measure Matching with Kernels for Generative Modeling

An Integrated Transportation Distance Between Kernels and Approximate Dynamic Risk Evaluation in Markov Systems

Kernel Wasserstein Distance

On the Probabilistic Approximation in Reproducing Kernel Hilbert Spaces

The Exploitation of Distance Distributions for Clustering

Probability error bounds for approximation of functions in reproducing kernel Hilbert spaces

An appointment with Reproducing Kernel Hilbert Space generated by Generalized Gaussian RBF as $L^2-$measure

Hierarchical Integral Probability Metrics: A distance on random probability measures with low sample complexity

A New Distance Measure for Hidden Markov Models

Hypothesis testing using pairwise distances and associated kernels (with Appendix)

Optimal Transport for Kernel Gaussian Mixture Models

A unified framework for multivariate two-sample and k-sample kernel-based quadratic distance goodness-of-fit tests

Semblance: A Rank-Based Kernel on Probability Spaces for Niche Detection

Entropy numbers of Reproducing Hilbert Space of zonal positive definite kernels on compact two-point homogeneous spaces

Hilbert Curve Projection Distance for Distribution Comparison