Abstract:A family of norm-based least squares support vector machines (LSSVMs) is the effective statistical learning tool and has attracted considerable attentions. However, there are two critical problems: (1) LSSVM is ill-conditioned or singular when the sample size is much less than feature number and thus causes over-fitting for small sample size (SSS) problem, while other norm-based LSSVMs own slower training speed and cannot deal with large scale data. (2) Norm-based LSSVMs pay less attention to the edge points that are important for learning the final classifier. To overcome the above drawbacks, by replacing $$ L_{2}\text{-}norm $$ with $$ L_{\infty }\text{-}norm $$ in empirical loss, we construct a robust classifier, named $$ L_{\infty }\text{-}LSSVM $$ in short. After adjustment, $$ L_{\infty }\text{-}LSSVM $$ can detect edge points effectively and enhances the capability of the robustness and the generalization. Also, inspired by the idea of sequential minimal optimization (SMO), we design a novel SMO-typed iterative algorithm. The new algorithm not only guarantees the convergence of optimum solution in theory, but also owns the lower computational time and storage space when data size is large. Finally, extensive numerical experiments validate the above opinions again on four groups of artificial data, Non-i.i.d data: fault detection of railway turnout, four SSS datasets, two types of massive datasets and six benchmark datasets.

Least Squares Support Vector Machine with Self-Organizing Multiple Kernel Learning and Sparsity.

Least Squares Support Vector Machine Regression Based on Sparse Samples and Mixture Kernel Learning.

Efficient Sparse Least Squares Support Vector Machines for Regression

Sparse Least Squares Support Vector Machine and Its Applications

Sparse Least Squares Support Vector Machine for Function Estimation

Cost-Constraint Based Multiple Kernel Least Squares Support Vector Machine and Its Application

Sparse multiple kernel for least square support vector regression

More Efficient Sparse Multi-kernel Based Least Square Support Vector Machine

A Robust Least Squares Support Vector Machine Based on L∞-norm

Improving the Solution of Least Squares Support Vector Machines with Application to A Blast Furnace System

Selective Ensemble Least Square Support Vector Machine With Its Application

Selective Recursive LSSVR and its Applications in Process Modeling

An Improved LSSVM Regression Algorithm

A New Sparse LSSVM Method Based the Revised LARS

Sparse Least Squares Low Rank Kernel Machines

Online Learning Algorithm For Lssvm Based Modeling With Time-Varying Kernels

Reconstruct the Support Vectors to Improve Lssvm Sparseness for Mill Load Prediction

Modeling method of least squares support vector regression based on vector base learning

Dynamic least squares support vector machine

A direct sparse algorithm for LS-SVR

Sparse Least Absolute Deviation Support Vector Machine