Abstract:Structural information is very important for improving the classification performance of classifiers. In order to increase the generalization performance of support vector machine (SVM) directly, several kinds of structured SVMs have been proposed. These algorithms with structural information only simply embed the global structural information or the local within-class information into SVM model. Thus, they sometimes are not suitable for real-world problems. To overcome the drawbacks, we firstly propose a novel SVM with local structural information (LSI-SVM) in this paper. In the LSI-SVM, the <i>K</i>-nearest neighbor (KNN) method is adopted. Applying the KNN method, the farthest neighbors set intra-class and the nearest neighbors set inter-class of the overall samples are obtained. It is more reasonable to maximize the margin between the nearest neighbors and the farthest neighbors. Both the global and local data structures are added into the optimization problem, making the LSI-SVM can fully utilize the underlying structural information and yield better performance. Furthermore, for nonlinear classification, the reproducing kernel Hilbert space theory is introduced and the kernel-based LSI-SVM is generated. What's more, in order to accelerate the training speed of LSI-SVM, a stochastic gradient LSI-SVM (LSI-SVM +) is constructed using the stochastic gradient descent (SGD) solver. Lastly, experimental results on regular-scale datasets, steel surface defects datasets, ORL face dataset and large-scale datasets demonstrate that both LSI-SVM and LSI-SVM + outperform other state-of-the-art algorithms on accuracy. In the meanwhile, our LSI-SVM + has high efficiency.

Optimal Locality Regularized Least Squares Support Vector Machine Via Alternating Optimization

Optimal Locality Preserving Least Square Support Vector Machine

Unified Locally Linear Classifiers with Diversity-Promoting Anchor Points

Locality Preserving Semi-Supervised Support Vector Machine.

A New Weighted Least Squares Support Vector Machines and Its Sequential Minimal Optimization Algorithm

Stochastic gradient support vector machine with local structural information for pattern recognition

Local Parameter Optimization of LSSVM for Industrial Soft Sensing with Big Data and Cloud Implementation

Fast Incremental Learning Method for One-Class Support Vector Machine

Structural Least Square Twin Support Vector Machine for Classification

Local Orthogonality Preserving Alignment for Nonlinear Dimensionality Reduction

Nonlinear Dimensionality Reduction by Local Orthogonality Preserving Alignment

Robust Least-Squares Support Vector Machine with Minimization of Mean and Variance of Modeling Error.

Fast Sparse Approximation for Least Squares Support Vector Machine.

Sparse Least Squares Support Vector Machine with L0-norm in Primal Space.

A Non-Biased Form of Least Squares Support Vector Classifier and Its Fast Online Learning

Optimal Regularization Parameters Selection for Laplacian Support Vector Machine

Robust least squares one-class support vector machine

An Improved LSSVM Regression Algorithm

Improved Sparse LSSVMS Based on the Localized Generalization Error Model

Single Sequential Minimal Optimization: an Improved SVMs Training Algorithm

A Fast Iterative Single Data Approach to Training Unconstrained Least Squares Support Vector Machines