Abstract:Structural information is very important for improving the classification performance of classifiers. In order to increase the generalization performance of support vector machine (SVM) directly, several kinds of structured SVMs have been proposed. These algorithms with structural information only simply embed the global structural information or the local within-class information into SVM model. Thus, they sometimes are not suitable for real-world problems. To overcome the drawbacks, we firstly propose a novel SVM with local structural information (LSI-SVM) in this paper. In the LSI-SVM, the <i>K</i>-nearest neighbor (KNN) method is adopted. Applying the KNN method, the farthest neighbors set intra-class and the nearest neighbors set inter-class of the overall samples are obtained. It is more reasonable to maximize the margin between the nearest neighbors and the farthest neighbors. Both the global and local data structures are added into the optimization problem, making the LSI-SVM can fully utilize the underlying structural information and yield better performance. Furthermore, for nonlinear classification, the reproducing kernel Hilbert space theory is introduced and the kernel-based LSI-SVM is generated. What's more, in order to accelerate the training speed of LSI-SVM, a stochastic gradient LSI-SVM (LSI-SVM +) is constructed using the stochastic gradient descent (SGD) solver. Lastly, experimental results on regular-scale datasets, steel surface defects datasets, ORL face dataset and large-scale datasets demonstrate that both LSI-SVM and LSI-SVM + outperform other state-of-the-art algorithms on accuracy. In the meanwhile, our LSI-SVM + has high efficiency.

A Supervised Local Linear Embedding Based SVM Text Classification Algorithm

Improving Text Classification Using Local Latent Semantic Indexing

A Local Online Learning Approach for Non-linear Data.

Supervised locally linear embedding with probability-based distance for classification

Unified Locally Linear Classifiers with Diversity-Promoting Anchor Points

Probability-Based Locally Linear Embedding for Classification

Stochastic gradient support vector machine with local structural information for pattern recognition

Local Feature Discriminant Projection

AN IMPROVED ALGORITHM ON LOCALLY LINEAR EMBEDDING

An Iterative Locally Linear Embedding Algorithm

Locally Linear SVMs Based on Boundary Anchor Points Encoding

Embedding Regularizer Learning for Multi-View Semi-Supervised Classification.

Feature Selection for Support Vector Machines in Text Categorization

A Joint Learning Framework for Optimal Feature Extraction and Multi-class SVM

Weighted Linear Embedding: Utilizing Local and Nonlocal Information Sufficiently

Chinese text classification based on character-level CNN and SVM

Local Relevance Weighted Maximum Margin Criterion for Text Classification.

Small World Neighborhood Optimized Local Linear Embedding Algorithm

Classification of Hyperspectral Remote Sensing Image Using Improved LS-SVM

Local Regression and Global Information-Embedded Dimension Reduction.

Large-scale support vector machine classification with redundant data reduction