Abstract:Locally Linear Support Vector Machine (LLSVM) has been actively used in classification tasks due to its capability of classifying nonlinear patterns. However, existing LLSVM suffers from two drawbacks: (1) a particular and appropriate regularization for LLSVM has not yet been addressed; (2) it usually adopts a three-stage learning scheme composed of learning anchor points by clustering, learning local coding coordinates by a predefined coding scheme, and finally learning for training classifiers. We argue that this decoupled approaches oversimplifies the original optimization problem, resulting in a large deviation due to the disparate purpose of each step. To address the first issue, we propose a novel diversified regularization which could capture infrequent patterns and reduce the model size without sacrificing the representation power. Based on this regularization, we develop a joint optimization algorithm among anchor points, local coding coordinates and classifiers to simultaneously minimize the overall classification risk, which is termed as Diversified and Unified Locally Linear Support Vector Machine(DU-LLSVM for short). To the best of our knowledge, DU-LLSVM is the first principled method that directly learns sparse local coding and can be easily generalized to other supervised learning models. Extensive experiments showed that DU-LLSVM consistently surpassed several state-of-the-art methods with a predefined local coding scheme (e.g. LLSVM) or a supervised anchor point learning(e.g. SAPL-LLSVM).

Optimal Locality Preserving Least Square Support Vector Machine

Optimal Locality Regularized Least Squares Support Vector Machine Via Alternating Optimization

Orthogonal Locality Preserving Indexing

Robust Least-Squares Support Vector Machine with Minimization of Mean and Variance of Modeling Error.

Unified Locally Linear Classifiers with Diversity-Promoting Anchor Points

Sparse Least Squares Support Vector Machine with L0-norm in Primal Space.

Robust least squares one-class support vector machine

A New Weighted Least Squares Support Vector Machines and Its Sequential Minimal Optimization Algorithm

Locality Preserving Semi-Supervised Support Vector Machine.

Least Squares Hidden Space Support Vector Machines

Asymmetric and robust loss function driven least squares support vector machine

Robust Least Squares Support Vector Machines with Applications to Soft-Sensing

Fast Sparse Approximation for Least Squares Support Vector Machine.

Improved Sparse LSSVMS Based on the Localized Generalization Error Model

Structural Least Square Twin Support Vector Machine for Classification

Sparse Hidden Space Support Vector Machine

The application of least squares support vector machine for classification

Recursive Robust Least Squares Support Vector Regression Based on Maximum Correntropy Criterion

Sparse Least Squares Low Rank Kernel Machines

Sparse Least Absolute Deviation Support Vector Machine

Efficient Sparse Least Squares Support Vector Machines for Regression