Abstract:Inspired by the recent work [28] on the statistical robustness of empirical risks in reproducing kernel Hilbert space (RKHS) where the training data are potentially perturbed or even corrupted, we take a step further in this paper to investigate the statistical robustness of the kernel learning estimator (the regularized empirical risk minimizer or stationary point). We begin by deriving qualitative statistical robustness of the estimator of the regularized empirical risk minimizer for a broad class of convex cost functions when all of the training data are potentially perturbed under some topological structures, and then move on to consider the quantitative statistical robustness of the stationary solution for a specific case that the cost function is continuously differentiable but not necessarily convex. In the latter case, we derive the first-order optimality condition of the regularized expected risk minimization problem, which is essentially a stochastic variational inequality problem (SVIP) in RKHS, and then use the SVIP as a platform to investigate local and global Lipschitz continuity of the stationary solution against perturbation of the probability distribution under the Fortet-Mourier metric. A crucial assumption in the analysis is that the perturbed data are independent and identically distributed (iid). In some practical applications, this assumption may not be fulfilled when a small proportion of perceived data is seriously perturbed/contaminated. In this case, we use the influence function to investigate the impact of single data perturbation on the expected risk minimizer. Differing from [64, Chapter 10], we concentrate on constrained expected risk minimization problems. The research is essentially down to the derivation of the implicit function theorem of the SVIP in RKHS. Finally, we illustrate our theoretical analysis with a couple of academic examples.

Local Learning Estimates by Integral Operators.

Local Linear M‐estimation in Non‐parametric Spatial Regression

Local convergence rates of the nonparametric least squares estimator with applications to transfer learning

Local Regularity for Minimizers of Integral Functionals

Local Boundedness for Minimizers of Convex Integral Functionals in Metric Measure Spaces

Nonparametric learning of kernels in nonlocal operators

Local Convergence of Inexact Methods under the Hölder Condition

Probability Density Estimation Based on Nonparametric Local Kernel Regression.

A local stochastic Lipschitz condition with application to Lasso for high dimensional generalized linear models

Learning Vector-valued Functions with Local Rademacher Complexity and Unlabeled Data

On the Concentration of the Minimizers of Empirical Risks

Learning nonlocal regularization operators

An expanded analysis of local fractional integral inequalities via generalized -convexity

Noninteractive Locally Private Learning of Linear Models via Polynomial Approximations

Distributed Algorithms for U-statistics-based Empirical Risk Minimization

Statistical Robustness of Kernel Learning Estimator with Respect to Data Perturbation

Analysis of Regularized Learning for Linear-functional Data in Banach Spaces

Local Risk Bounds for Statistical Aggregation

Nearly Optimal $L_p$ Risk Minimization

l_inf-approximation of localized distributions

Theoretical Analysis of Divide-and-Conquer ERM: Beyond Square Loss and RKHS.