Abstract:Sparse support vector regression (SSVR) is an effective regression technique. It has been successfully applied to many practical problems. However, it remains challenging to handle the large-scale problems. A nice property of SSVR is double sparsity in the sense that most irrelevant features and samples have no effect on the regressor. Inspired by it, we propose a simultaneous safe feature and sample screening rule based on the strong convexity of objective function (FSSR1) to accelerate SSVR, including the linear and nonlinear cases. Most inactive features and samples can be simultaneously discarded before training SSVR. Only one reduced SSVR (RSSVR) needs to be solved. Moreover, to further speed up RSSVR, the screening rule based on duality gap (FSSR2) is applied to continuously discard the remaining inactive variables during the reduced model training process. Combining the rule with the grid-search method, the framework of our final method (FSSR-SSVR) is to alternatively conduct FSSR1 and FSSR2, which leads to substantial savings in both the memory usage and the computational cost. There are two appealing advantages of our FSSR-SSVR: first, it is safe, i.e., the features and samples discarded by FSSR are guaranteed to be inactive; second, it has synergy effect (in other words, the results of the previous feature screening can improve the performance of the next sample screening, and vice versa). Furthermore, the stochastic dual coordinate ascent (SDCA) method is employed as an efficient solver. Experiments on three synthetic datasets and 11 real-world datasets demonstrate the efficiency of our method.

∊-Kernel-free Soft Quadratic Surface Support Vector Regression

Kernel-Free Quadratic Surface Support Vector Regression with Non-Negative Constraints.

Quadratic Hyper-Surface Kernel-Free Least Squares Support Vector Regression

Flatheaded support vector machine for regression

Kernel-free Reduced Quadratic Surface Support Vector Machine with 0-1 Loss Function and L _p -norm Regularization

Nonlinear kernel-free quadratic hyper-surface support vector machine with 0-1 loss function

Kernel-Free Quadratic Surface Regression for Multi-Class Classification

A Kernel-Free Fuzzy Reduced Quadratic Surface Ν-Support Vector Machine with Applications

Sparse Universum Quadratic Surface Support Vector Machine Models for Binary Classification

Mini-batch Quasi-Newton Optimization for Large Scale Linear Support Vector Regression

Supervised Feature Selection via Quadratic Surface Regression with l_2,1 -Norm Regularization

A kernel-free L1 norm regularized ν-support vector machine model with application

Least Square Generalized Support Vector Machines for Regression

Convex Support Vector Regression

A kernel-free L1 norm regularized -support vector machine model with application

Sparse Least Squares Support Vector Machine for Function Estimation

Simplification of SMO Algorithm and Its Application in Solving Ε-Svr with Non-Positive Kernels

Kernel-Free Quadratic Surface Minimax Probability Machine for a Binary Classification Problem

Semismooth Newton Support Vector Machine

Simultaneous Safe Feature and Sample Elimination for Sparse Support Vector Regression.

Svr Plus Rvr: A Robust Sparse Kernel Method For Regression