Abstract:Inspired by the recent work [28] on the statistical robustness of empirical risks in reproducing kernel Hilbert space (RKHS) where the training data are potentially perturbed or even corrupted, we take a step further in this paper to investigate the statistical robustness of the kernel learning estimator (the regularized empirical risk minimizer or stationary point). We begin by deriving qualitative statistical robustness of the estimator of the regularized empirical risk minimizer for a broad class of convex cost functions when all of the training data are potentially perturbed under some topological structures, and then move on to consider the quantitative statistical robustness of the stationary solution for a specific case that the cost function is continuously differentiable but not necessarily convex. In the latter case, we derive the first-order optimality condition of the regularized expected risk minimization problem, which is essentially a stochastic variational inequality problem (SVIP) in RKHS, and then use the SVIP as a platform to investigate local and global Lipschitz continuity of the stationary solution against perturbation of the probability distribution under the Fortet-Mourier metric. A crucial assumption in the analysis is that the perturbed data are independent and identically distributed (iid). In some practical applications, this assumption may not be fulfilled when a small proportion of perceived data is seriously perturbed/contaminated. In this case, we use the influence function to investigate the impact of single data perturbation on the expected risk minimizer. Differing from [64, Chapter 10], we concentrate on constrained expected risk minimization problems. The research is essentially down to the derivation of the implicit function theorem of the SVIP in RKHS. Finally, we illustrate our theoretical analysis with a couple of academic examples.

Analytic perturbations and systematic bias in statistical modeling and inference

Inference in Perturbation Models, Finite Mixtures and Scan Statistics: The Volume-of-Tube Formula

Quantitative Analysis for Perturbed Abstract Inequality Systems in Banach Spaces

Statistical Robustness of Kernel Learning Estimator with Respect to Data Perturbation

Models as Approximations I: Consequences Illustrated with Linear Regression

Covariance constraints for stochastic inverse problems of computer models

A way to model stochastic perturbations in population dynamics models with bounded realizations

Perturbation Analysis of Singular Linear Systems with Arbitrary Index

Choosing observation operators to mitigate model error in Bayesian inverse problems

A Perturbation Method for Inference on Regularized Regression Estimates

Invariant subspaces of analytic perturbations

Local Perturbation Analysis of Functional Regression with Applications

Assessing Robustness of Machine Learning Models using Covariate Perturbations

Misspecified Bernstein-Von Mises theorem for hierarchical models

Bias Correction and Robust Inference in Semiparametric Models

Distributional Theory and Statistical Inference for Linear Functions of Eigenvectors with Small Eigengaps

Singular Perturbation: When the Perturbation Parameter Becomes a State-Dependent Function

Perturbation analysis of nonlinear matrix population models

Perturbation of error bounds

On the prevalence of information inconsistency in normal linear models

Empirical Perturbation Analysis of Linear System Solvers from a Data Poisoning Perspective