Abstract:Locally Linear Support Vector Machine (LLSVM) has been actively used in classification tasks due to its capability of classifying nonlinear patterns. However, existing LLSVM suffers from two drawbacks: (1) a particular and appropriate regularization for LLSVM has not yet been addressed; (2) it usually adopts a three-stage learning scheme composed of learning anchor points by clustering, learning local coding coordinates by a predefined coding scheme, and finally learning for training classifiers. We argue that this decoupled approaches oversimplifies the original optimization problem, resulting in a large deviation due to the disparate purpose of each step. To address the first issue, we propose a novel diversified regularization which could capture infrequent patterns and reduce the model size without sacrificing the representation power. Based on this regularization, we develop a joint optimization algorithm among anchor points, local coding coordinates and classifiers to simultaneously minimize the overall classification risk, which is termed as Diversified and Unified Locally Linear Support Vector Machine(DU-LLSVM for short). To the best of our knowledge, DU-LLSVM is the first principled method that directly learns sparse local coding and can be easily generalized to other supervised learning models. Extensive experiments showed that DU-LLSVM consistently surpassed several state-of-the-art methods with a predefined local coding scheme (e.g. LLSVM) or a supervised anchor point learning(e.g. SAPL-LLSVM).

An Improvement Algorithm of Doubly Regularized Support Vector Machine

Weighted Support Vector Machines for Regression and Its Application

A Novel Robust Smooth Support Vector Machine

Flatheaded support vector machine for regression

Improved Weighted Least Squares Support Vector Machines Algorithm And Its Applications In Spectroscopic Quantitative Analysis

One Kind of Reduced Learning Algorithm for Support Vector Domain Description

An Incremental Updating Method for Support Vector Machines

Robust Iterative Algorithm of Weighted Least Squares Support Vector Machine and Its Application in Spectral Analysis

L1-Norm Least Squares Support Vector Regression Via the Alternating Direction Method of Multipliers

Unified Locally Linear Classifiers with Diversity-Promoting Anchor Points

Mini-batch Quasi-Newton Optimization for Large Scale Linear Support Vector Regression

Improvements to Train Support Vector Machines Based on Convex Set Conception

Non-smooth optimization algorithm to solve the LINEX soft support vector machine

Improved 2-norm Based Fuzzy Least Squares Twin Support Vector Machine

L2,1-Norm Regularized Robust and Sparse Linear Discriminant Analysis via an Alternating Direction Method of Multipliers

On-line Support Vector Machine Training Algorithm and Its Application

New method for solving Ivanov regularization-based support vector machine learning

Linearized alternating direction method of multipliers for elastic-net support vector machines

Improving Sparsity and Scalability in Regularized Nonconvex Truncated-Loss Learning Problems

An unbiased LSSVM model for classification and regression

Sparse Norm-Regularized Reconstructive Coefficients Learning