Abstract:Inspired by the recent work [28] on the statistical robustness of empirical risks in reproducing kernel Hilbert space (RKHS) where the training data are potentially perturbed or even corrupted, we take a step further in this paper to investigate the statistical robustness of the kernel learning estimator (the regularized empirical risk minimizer or stationary point). We begin by deriving qualitative statistical robustness of the estimator of the regularized empirical risk minimizer for a broad class of convex cost functions when all of the training data are potentially perturbed under some topological structures, and then move on to consider the quantitative statistical robustness of the stationary solution for a specific case that the cost function is continuously differentiable but not necessarily convex. In the latter case, we derive the first-order optimality condition of the regularized expected risk minimization problem, which is essentially a stochastic variational inequality problem (SVIP) in RKHS, and then use the SVIP as a platform to investigate local and global Lipschitz continuity of the stationary solution against perturbation of the probability distribution under the Fortet-Mourier metric. A crucial assumption in the analysis is that the perturbed data are independent and identically distributed (iid). In some practical applications, this assumption may not be fulfilled when a small proportion of perceived data is seriously perturbed/contaminated. In this case, we use the influence function to investigate the impact of single data perturbation on the expected risk minimizer. Differing from [64, Chapter 10], we concentrate on constrained expected risk minimization problems. The research is essentially down to the derivation of the implicit function theorem of the SVIP in RKHS. Finally, we illustrate our theoretical analysis with a couple of academic examples.

Risk-sensitive Loss in Kernel Space for Robust Adaptive Filtering

A Kernel Function Optimization And Selection Algorithm Based On Cost Function Maximization

Kernel Matrix Learning with a General Regularized Risk Functional Criterion

Robust Kernel Recursive Adaptive Filtering Algorithms Based on M-estimate

Gradient Descent for Robust Kernel-Based Regression

Robust Regularized Kernel Regression.

Statistical Robustness of Kernel Learning Estimator with Respect to Data Perturbation

A quantized minimum kernel risk hyperbolic secant adaptive filtering algorithm

Robust cost-sensitive kernel method with Blinex loss and its applications in credit risk evaluation

Risk-Sensitive Extended Kalman Filter

Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning

An Improved Robust Kernel Adaptive Filtering Method for Time-Series Prediction

A Robust Adaptive FIR Filter Based on M-Estimation

Risk Sensitive Filtering with Randomly Delayed Measurements

Statistical Robustness of Empirical Risks in Machine Learning

Adaptive Multikernel Size-Based Maximum Correntropy Cubature Kalman Filter for the Robust State Estimation

A Kernel Normalized Data-Reusing Generalized Maximum Correntropy Algorithm

An Alternate View on Optimal Filtering in an RKHS

Quantized criterion-based kernel recursive least squares adaptive filtering for time series prediction

Minimum Cauchy Kernel Loss Based Robust Cubature Kalman Filter and Its Low Complexity Cost Version With Application on INS/OD Integrated Navigation System

Correntropy Induced Metric Penalized Sparse RLS Algorithm to Improve Adaptive System Identification