Abstract:In this paper, we present a unified approach to function approximation in reproducing kernel Hilbert spaces (RKHS) that establishes a previously unrecognized optimality property for several well-known function approximation techniques, such as minimum-norm interpolation, smoothing splines, and pseudo-inverses. We consider the problem of approximating a function belonging to an arbitrary real-valued RKHS on R^d based on approximate observations of the function. The observations are approximate in the sense that the actual observations (i.e., the true function values) are known only to belong to a convex set of admissible observations. We seek a minimax optimal approximation for the function that minimizes the supremum of the RKHS norm on the error between the true function and the chosen approximation subject only to the conditions that the true function belongs to a uniformly bounded uncertainty set of functions that satisfy the constraints on the observations and that the approximation is a member of the RKHS. We refer to such a solution as a minimax robust reconstruction. We characterize the solution to the minimax robust reconstruction problem and show that it is equivalent to solving a straightforward convex optimization problem. We demonstrate that a minimax robust reconstruction will generally be more stable than an approximation based on interpolation through a nominal set of observations and that, subject to some mild regularity conditions on the convex set of admissible observations, the minimax robust reconstruction is unconditionally stable. We motivate our results by characterizing the minimax robust reconstruction for several specific convex observational models and discuss relationships with other approaches to function approximation.

Equivalence between constrained optimal smoothing and Bayesian estimation

Error Bounds for a Kernel-Based Constrained Optimal Smoothing Approximation

Approximate Bayesian Computation for Smoothing

Nonlinearly Constrained Best Approximation in Hilbert Spaces: the Strong CHIP and the Basic Constraint Qualification.

A generalised smoothing approach for continuous, planar, inverse kinematics problems

Kernel Bayesian Inference with Posterior Regularization

Optimal score estimation via empirical Bayes smoothing

Minimax Robust Function Reconstruction in Reproducing Kernel Hilbert Spaces

A variable smoothing for Nonconvexly constrained nonsmooth optimization with application to sparse spectral clustering

Posterior linearisation smoothing with robust iterations

Maximum likelihood smoothing estimation in state-space models: An incomplete-information based approach

A stochastic optimisation unadjusted Langevin method for empirical Bayesian estimation in semi-blind image deblurring problems

Ensemble transport smoothing. Part I: Unified framework

Optimal approximating Markov chains for Bayesian inference

A stochastic moving ball approximation method for smooth convex constrained minimization

Extending error bounds for radial basis function interpolation to measuring the error in higher order Sobolev norms

Generalized Kalman Smoothing: Modeling and Algorithms

Approximation of optimization problems with constraints through kernel Sum-of-Squares

Ideal Bayesian Spatial Adaptation

Convergence of the Adapted Smoothed Empirical Measures