Abstract:In this paper, we present a unified approach to function approximation in reproducing kernel Hilbert spaces (RKHS) that establishes a previously unrecognized optimality property for several well-known function approximation techniques, such as minimum-norm interpolation, smoothing splines, and pseudo-inverses. We consider the problem of approximating a function belonging to an arbitrary real-valued RKHS on R^d based on approximate observations of the function. The observations are approximate in the sense that the actual observations (i.e., the true function values) are known only to belong to a convex set of admissible observations. We seek a minimax optimal approximation for the function that minimizes the supremum of the RKHS norm on the error between the true function and the chosen approximation subject only to the conditions that the true function belongs to a uniformly bounded uncertainty set of functions that satisfy the constraints on the observations and that the approximation is a member of the RKHS. We refer to such a solution as a minimax robust reconstruction. We characterize the solution to the minimax robust reconstruction problem and show that it is equivalent to solving a straightforward convex optimization problem. We demonstrate that a minimax robust reconstruction will generally be more stable than an approximation based on interpolation through a nominal set of observations and that, subject to some mild regularity conditions on the convex set of admissible observations, the minimax robust reconstruction is unconditionally stable. We motivate our results by characterizing the minimax robust reconstruction for several specific convex observational models and discuss relationships with other approaches to function approximation.

Minimax Robust Function Reconstruction in Reproducing Kernel Hilbert Spaces

Almost-sure quasi-optimal approximation in reproducing kernel Hilbert spaces

Error Bounds for a Kernel-Based Constrained Optimal Smoothing Approximation

Sparse multiresolution representations with adaptive kernels

Extending error bounds for radial basis function interpolation to measuring the error in higher order Sobolev norms

Approximation of RKHS Functionals by Neural Networks

On the Approximation of Kernel functions

A new kernel method for the uniform approximation in reproducing kernel Hilbert spaces

Approximation Algorithms for Min-max-min Robust Optimization and K-Adaptability under Objective Uncertainty

Duality in RKHSs with Infinite Dimensional Outputs: Application to Robust Losses

On the Probabilistic Approximation in Reproducing Kernel Hilbert Spaces

The art of BART: Minimax optimality over nonhomogeneous smoothness in high dimension

Gradient Descent for Robust Kernel-Based Regression

Adaptive estimation for nonlinear systems using reproducing kernel Hilbert spaces

On the Optimality of Misspecified Kernel Ridge Regression

Optimality of Robust Online Learning

Generalization Properties of Hyper-Rkhs and Its Applications

Analysis of Regularized Least-Squares in Reproducing Kernel Kreĭn Spaces

Approximating Reproducing Kernel Hilbert Space Functions by Bernstein Operators

An Alternate View on Optimal Filtering in an RKHS