Abstract:Reproducing kernel Hilbert spaces (RKHSs) are key elements of many non-parametric tools successfully used in signal processing, statistics, and machine learning. In this work, we aim to address three issues of the classical RKHS based techniques. First, they require the RKHS to be known a priori, which is unrealistic in many applications. Furthermore, the choice of RKHS affects the shape and smoothness of the solution, thus impacting its performance. Second, RKHSs are ill-equipped to deal with heterogeneous degrees of smoothness, i.e., with functions that are smooth in some parts of their domain but vary rapidly in others. Finally, the computational complexity of evaluating the solution of these methods grows with the number of data points, rendering these techniques infeasible for many applications. Though kernel learning, local kernel adaptation, and sparsity have been used to address these issues, many of these approaches are computationally intensive or forgo optimality guarantees. We tackle these problems by leveraging a novel integral representation of functions in RKHSs that allows for arbitrary centers and different kernels at each center. To address the complexity issues, we then write the function estimation problem as a sparse functional program that explicitly minimizes the support of the representation leading to low complexity solutions. Despite their non-convexity and infinite dimensionality, we show these problems can be solved exactly and efficiently by leveraging duality, and we illustrate this new approach in simulated and real data.

Reproducing kernel Banach spaces for machine learning

Vector-valued Reproducing Kernel Banach Spaces with Applications to Multi-task Learning

Mirror Descent on Reproducing Kernel Banach Spaces

Generalization Properties of Hyper-Rkhs and Its Applications

Approximate Representer Theorems in Non-reflexive Banach Spaces

Sparse multiresolution representations with adaptive kernels

Nonuniform sampling, reproducing kernels, and the associated Hilbert spaces

An algebra structure for reproducing kernel Hilbert spaces

Neural reproducing kernel Banach spaces and representer theorems for deep networks

Regularized Regression Problem in hyper-RKHS for Learning Kernels

On the stability test for reproducing kernel Hilbert spaces

Learning in RKHM: a $C^*$-Algebraic Twist for Kernel Machines

Duality in RKHSs with Infinite Dimensional Outputs: Application to Robust Losses

Reproducing Kernel Hilbert Space, Mercer's Theorem, Eigenfunctions, Nyström Method, and Use of Kernels in Machine Learning: Tutorial and Survey

Analysis of Regularized Least-Squares in Reproducing Kernel Kreĭn Spaces

Analysis of Least Squares Regularized Regression in Reproducing Kernel Krein Spaces

A Primer on Reproducing Kernel Hilbert Spaces

A refinement of the stability test for reproducing kernel Hilbert spaces

An appointment with Reproducing Kernel Hilbert Space generated by Generalized Gaussian RBF as $L^2-$measure

Reproducing kernels and choices of associated feature spaces, in the form of $L^{2}$-spaces

Regularization of Hidden Markov Models Embedded into Reproducing Kernel Hilbert Space