Abstract:<p>Robust least squares support vector regression (RLSSVR), minimizing the variance and mean of the global modeling errors, has achieved the excellent performance in dealing with outliers. However, generalizing the RLSSVR for solving the binary classification problems is easily misguided by the outliers because the differences in the modeling errors of the different classes are not considered. To address this issue, a robust least squares support vector classifier (RLSSVC) with optimal error distribution is proposed. RLSSVC minimizes the mean and variance of the modeling errors class-wisely, and considers the difference in the modeling errors of the different classes. Specifically, the binary classification problems are considered at first, the variance analysis indicates that the variance of the modeling errors of RLSSVC is smaller than that of RLSSVR. According to the validity in solving the binary classification problems, RLSSVC is naturally generalized for solving the multiclass classification problems by introducing multiple error adjusting factors. The robustness analysis provides a theoretical guarantee for the robustness of RLSSVC, which delivers that RLSSVC assigns the smaller weights for the training instances with the larger errors, while the larger weights for the training instances with the smaller errors. Furthermore, our optimization objective function is strictly convex and thus can obtain their corresponding closed-form solutions, resulting in higher computational performance. Finally, the performance of RLSSVC is further improved by introducing the metric learning and kernel trick. Theoretical and experimental results indicate that the proposed RLSSVC achieves the better classification effect with the lower computational costs.</p>

Convex Surrogate Minimization in Classification

On the Rates of Convergence from Surrogate Risk Minimizers to the Bayes Optimal Classifier.

Inference with non-differentiable surrogate loss in a general high-dimensional classification framework

Binary Excess Risk for Smooth Convex Surrogates.

GEV Regression with Convex Loss Applied to Imbalanced Binary Classification.

The Optimality of Kernel Classifiers in Sobolev Space

Supervised classification via minimax probabilistic transformations

Solving Nonsmooth Nonconvex Compound Stochastic Programs with Applications to Risk Measure Minimization

Minimax risk classifiers with 0-1 loss

Robust multiclass least squares support vector classifier with optimal error distribution

Convex optimization over a probability simplex

Surrogate Maximization/minimization Algorithms and Extensions

Learning Surrogate Losses

Non-asymptotic Excess Risk Bounds for Classification with Deep Convolutional Neural Networks

Convex Support Vector Regression

Misclassification excess risk bounds for PAC-Bayesian classification via convexified loss

Constrained Classification and Policy Learning

Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators

Kernel Based Simple Regularized Multiple Criteria Linear Programs for Binary Classification

Risk-Based Robust Statistical Learning by Stochastic Difference-of-Convex Value-Function Optimization

Multivariate Robust Second-Order Stochastic Dominance and Resulting Risk-Averse Optimization