Abstract:Reproducing kernel Hilbert spaces (RKHSs) are key elements of many non-parametric tools successfully used in signal processing, statistics, and machine learning. In this work, we aim to address three issues of the classical RKHS based techniques. First, they require the RKHS to be known a priori, which is unrealistic in many applications. Furthermore, the choice of RKHS affects the shape and smoothness of the solution, thus impacting its performance. Second, RKHSs are ill-equipped to deal with heterogeneous degrees of smoothness, i.e., with functions that are smooth in some parts of their domain but vary rapidly in others. Finally, the computational complexity of evaluating the solution of these methods grows with the number of data points, rendering these techniques infeasible for many applications. Though kernel learning, local kernel adaptation, and sparsity have been used to address these issues, many of these approaches are computationally intensive or forgo optimality guarantees. We tackle these problems by leveraging a novel integral representation of functions in RKHSs that allows for arbitrary centers and different kernels at each center. To address the complexity issues, we then write the function estimation problem as a sparse functional program that explicitly minimizes the support of the representation leading to low complexity solutions. Despite their non-convexity and infinite dimensionality, we show these problems can be solved exactly and efficiently by leveraging duality, and we illustrate this new approach in simulated and real data.

Positive Definite Kernels, Algorithms, Frames, and Approximations

Hilbert space valued Gaussian processes, their kernels, factorizations, and covariance structure

Cross-validatory Framework for Optimal Parameter Estimation of KPCA and KPLS Models

An Infinite Dimensional Analysis of Kernel Principal Components

New characterizations of reproducing kernel Hilbert spaces and applications to metric geometry

Positive definite kernels and boundary spaces

Tropical Reproducing Kernels and Optimization

Nonuniform sampling, reproducing kernels, and the associated Hilbert spaces

Product kernels are efficient and flexible tools for high-dimensional scattered data interpolation

Operator-Valued Kernels, Machine Learning, and Dynamical Systems

Canonical data-reconstructions via kernels, Hilbert space-valued Gaussian processes, and quantum states

Paired Kernels and Their Applications

Full-rank orthonormal bases for conditionally positive definite kernel-based spaces

An empirical approach for probing the definiteness of kernels

Positive Definite Kernels and Random Sequences Connected to Polynomial Hypergroups

Unified Native Spaces in Kernel Methods

Kernel Methods in Hyperbolic Spaces

Matrix-Based Kernel Subspace Methods

Sparse multiresolution representations with adaptive kernels

Kernel Optimisation for KPCA Based on Gaussianity Estimation.

On the Approximation of Kernel functions