Abstract:The truncated regular L-1-loss support vector machine can eliminate the excessive number of support vectors (SVs); thus, it has significant advantages in robustness and scalability. However, in this paper, we discover that the associated state-of-the-art solvers, such as difference convex algorithm and concave-convex procedure, not only have limited sparsity promoting property for general truncated losses especially the L-2-loss but also have poor scalability for large-scale problems. To circumvent these drawbacks, we present a general multistage scheme with explicit interpretation regarding SVs as well as outliers. In particular, we solve the general nonconvex truncated loss minimization through a sequence of associated convex subproblems, in which the outliers are removed in advance. The proposed algorithm can be regarded as a structural optimization attempt carefully considering sparsity imposed by the nonconvex truncated losses. We show that this general multistage algorithm offers sufficient sparsity especially for the truncated L-2-loss. To further improve the scalability, we propose a linear multistep algorithm by employing a single iteration of coordinate descent to monotonically decrease the objective function at each stage and a kernel algorithm by using the Karush-Kuhn-Tucker conditions to cheaply find most part of the outliers for the next stage. Comparison experiments demonstrate that our methods have superiority in sparsity as well as efficiency in scalability.

Sparse Recovery Via Nonconvex Regularized M-estimators over ℓ_q-Balls

Stability and robustness of the l2/lq-minimization for block sparse recovery.

Sparse Estimation Via ℓ_q Optimization Method in High-Dimensional Linear Regression

Efficient Sparse Recovery Via Adaptive Non-Convex Regularizers with Oracle Property

Robust Sparse Recovery Via Non-Convex Optimization

A Non-convex Approach for Sparse Recovery with Convergence Guarantee

Low-rank matrix recovery via nonconvex optimization methods with application to errors-in-variables matrix regression

The Convergence Guarantees of a Non-Convex Approach for Sparse Recovery.

Sparse Signal Recovery Via Minimax-Concave Penalty and ℓ 1 -Norm Loss Function.

From Least Squares to Sparse: A Non-Convex Approach with Guarantee

Local and Global Optimality of LP Minimization for Sparse Recovery.

Robust Sparse Recovery Via Weakly Convex Optimization in Impulsive Noise

Efficient and Robust Recovery of Sparse Signal and Image Using Generalized Nonconvex Regularization

Fast Sparse Recovery Via Non-Convex Optimization

A Robust Recovery Algorithm with Smoothing Strategies.

Relaxed Sparse Eigenvalue Conditions for Sparse Estimation Via Non-Convex Regularized Regression

Sufficient Conditions for Sparse Recovery by Weighted ℓ1-Constrained Quadratic Programming.

The ℓ2, Regularized Group Sparse Optimization: Lower Bound Theory, Recovery Bound and Algorithms

Improving Sparsity and Scalability in Regularized Nonconvex Truncated-Loss Learning Problems

Matrix recovery from nonconvex regularized least absolute deviations

Improving Spiking Sparse Recovery via Non-Convex Penalties