Abstract:Reproducing kernel Hilbert spaces (RKHSs) are key elements of many non-parametric tools successfully used in signal processing, statistics, and machine learning. In this work, we aim to address three issues of the classical RKHS based techniques. First, they require the RKHS to be known a priori, which is unrealistic in many applications. Furthermore, the choice of RKHS affects the shape and smoothness of the solution, thus impacting its performance. Second, RKHSs are ill-equipped to deal with heterogeneous degrees of smoothness, i.e., with functions that are smooth in some parts of their domain but vary rapidly in others. Finally, the computational complexity of evaluating the solution of these methods grows with the number of data points, rendering these techniques infeasible for many applications. Though kernel learning, local kernel adaptation, and sparsity have been used to address these issues, many of these approaches are computationally intensive or forgo optimality guarantees. We tackle these problems by leveraging a novel integral representation of functions in RKHSs that allows for arbitrary centers and different kernels at each center. To address the complexity issues, we then write the function estimation problem as a sparse functional program that explicitly minimizes the support of the representation leading to low complexity solutions. Despite their non-convexity and infinite dimensionality, we show these problems can be solved exactly and efficiently by leveraging duality, and we illustrate this new approach in simulated and real data.

Full-rank orthonormal bases for conditionally positive definite kernel-based spaces

Positive Definite Kernels, Algorithms, Frames, and Approximations

Variations of Orthonormal Basis Matrices of Subspaces

On Quasi-Localized Dual Pairs in Reproducing Kernel Hilbert Spaces

Estimating the Condition Number for Multivariate Interpolation Problems

Invariant kernels on Riemannian symmetric spaces: a harmonic-analytic approach

Scattered Data Interpolation on Embedded Submanifolds with Restricted Positive Definite Kernels: Sobolev Error Estimates

Dynamics of Orthonormal Bases Associated to Basins of Attraction

Nonuniform sampling, reproducing kernels, and the associated Hilbert spaces

Approximation beats concentration? An approximation view on inference with smooth radial kernels

Product kernels are efficient and flexible tools for high-dimensional scattered data interpolation

Data-Driven Linear Complexity Low-Rank Approximation of General Kernel Matrices: A Geometric Approach

Low-Rank Kernel Matrix Approximation Using Skeletonized Interpolation With Endo- or Exo-Vertices

The condition of gram matrices and related problems

An empirical approach for probing the definiteness of kernels

An Adaptive Factorized Nyström Preconditioner for Regularized Kernel Matrices

A new kernel method for the uniform approximation in reproducing kernel Hilbert spaces

Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices

Entropy numbers of Reproducing Hilbert Space of zonal positive definite kernels on compact two-point homogeneous spaces

On the numerical rank of radial basis function kernels in high dimension

Sparse multiresolution representations with adaptive kernels