Abstract:As enjoying the closed form solution, least squares support vector machine (LSSVM) has been widely used for classification and regression problems having the comparable performance with other types of SVMs. However, LSSVM has two drawbacks: sensitive to outliers and lacking sparseness. Robust LSSVM (R-LSSVM) overcomes the first partly via nonconvex truncated loss function, but the current algorithms for R-LSSVM with the dense solution are faced with the second drawback and are inefficient for training large-scale problems. In this paper, we interpret the robustness of R-LSSVM from a re-weighted viewpoint and give a primal R-LSSVM by the representer theorem. The new model may have sparse solution if the corresponding kernel matrix has low rank. Then approximating the kernel matrix by a low-rank matrix and smoothing the loss function by entropy penalty function, we propose a convergent sparse R-LSSVM (SR-LSSVM) algorithm to achieve the sparse solution of primal R-LSSVM, which overcomes two drawbacks of LSSVM simultaneously. The proposed algorithm has lower complexity than the existing algorithms and is very efficient for training large-scale problems. Many experimental results illustrate that SR-LSSVM can achieve better or comparable performance with less training time than related algorithms, especially for training large scale problems.

Sparse multiple kernel for least square support vector regression

Sparse Least Squares Support Vector Machine for Function Estimation

Least Squares Support Vector Machine Regression Based on Sparse Samples and Mixture Kernel Learning.

Efficient Sparse Least Squares Support Vector Machines for Regression

More Efficient Sparse Multi-kernel Based Least Square Support Vector Machine

Sparse Least Squares Low Rank Kernel Machines

Least Squares Support Vector Machine with Self-Organizing Multiple Kernel Learning and Sparsity.

Online Kernel Learning with a Near Optimal Sparsity Bound

A direct sparse algorithm for LS-SVR

Cost-Constraint Based Multiple Kernel Least Squares Support Vector Machine and Its Application

Fast Sparse Approximation for Least Squares Support Vector Machine.

Sparse Data-Dependent Kernel Principal Component Analysis Based on Least Squares Support Vector Machine for Feature Extraction and Recognition.

Non-parametric Least Square Support Vector Machine

Robust Least Squares Projection Twin Support Vector Machine and Its Sparse Solution

Sparse Least Absolute Deviation Support Vector Machine

Sparse Kernel Feature Extraction Via Support Vector Learning.

Adaptive Pruning Algorithm for Least Squares Support Vector Machine Classifier

A Learning Approach to Derive Sparse Kernel Minimum Square Error Model

Sparse Least Squares Support Vector Machine and Its Applications

Svr Plus Rvr: A Robust Sparse Kernel Method For Regression

Sparse Algorithm for Robust LSSVM in Primal Space