Abstract:Inspired by the recent work [28] on the statistical robustness of empirical risks in reproducing kernel Hilbert space (RKHS) where the training data are potentially perturbed or even corrupted, we take a step further in this paper to investigate the statistical robustness of the kernel learning estimator (the regularized empirical risk minimizer or stationary point). We begin by deriving qualitative statistical robustness of the estimator of the regularized empirical risk minimizer for a broad class of convex cost functions when all of the training data are potentially perturbed under some topological structures, and then move on to consider the quantitative statistical robustness of the stationary solution for a specific case that the cost function is continuously differentiable but not necessarily convex. In the latter case, we derive the first-order optimality condition of the regularized expected risk minimization problem, which is essentially a stochastic variational inequality problem (SVIP) in RKHS, and then use the SVIP as a platform to investigate local and global Lipschitz continuity of the stationary solution against perturbation of the probability distribution under the Fortet-Mourier metric. A crucial assumption in the analysis is that the perturbed data are independent and identically distributed (iid). In some practical applications, this assumption may not be fulfilled when a small proportion of perceived data is seriously perturbed/contaminated. In this case, we use the influence function to investigate the impact of single data perturbation on the expected risk minimizer. Differing from [64, Chapter 10], we concentrate on constrained expected risk minimization problems. The research is essentially down to the derivation of the implicit function theorem of the SVIP in RKHS. Finally, we illustrate our theoretical analysis with a couple of academic examples.

Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Non-Asymptotic Guarantees for Robust Statistical Learning under (1+ε)-th Moment Assumption

Regularization and variable selection for infinite variance autoregressive models

Asymptotic expansion for nonparametric M-estimator in a nonlinear regression model with long-memory errors

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

Robust Nonparametric Regression with Deep Neural Networks

High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality

Robust Elastic Net Regression

Asymptotic equivalence and adaptive estimation for robust nonparametric regression

Asymptotics of Estimates in Constrained Nonlinear Regression with Long-Range Dependent Innovations

Computationally Efficient and Statistically Optimal Robust High-Dimensional Linear Regression

When can Regression-Adjusted Control Variates Help? Rare Events, Sobolev Embedding and Minimax Optimality

Statistical Robustness of Kernel Learning Estimator with Respect to Data Perturbation

Robust Fused Lasso Penalized Huber Regression with Nonasymptotic Property and Implementation Studies

Existence of solutions to the nonlinear equations characterizing the precise error of M-estimators

Robust Wasserstein Optimization and its Application in Mean-CVaR

Asymptotic Normality and Confidence Intervals for Prediction Risk of the Min-Norm Least Squares Estimator.

Outlier-robust sparse/low-rank least-squares regression and robust matrix completion

Robust and sparse estimation methods for high dimensional linear and logistic regression

Inference for Conditional Value-at-risk of a Predictive Regression