Abstract:The random weight idea in neural network has existed for long time and attracted wide attention for its faster learning speed than that of tuning-based methods. Recently, extreme learning machine (ELM), one non-tuning algorithm for single hidden layer neural network (SLFN) was proposed and applied in many applications. ELM can learn hundreds of times faster than that of the traditional tuning-based methods while obtaining better generalization performance. But since the ELM randomly assigns input weights, its stability and accuracy will decline sharply for high-dimension and small-sample dataset. Although dimension reduction can be used to improve it, this two-stage method will need more physical space and increase structure complexity of the model. In this paper, we propose a novel single-stage algorithm, base projection vector machine (BPVM) and its kernel version—kernel projection vector machine (KPVM) by combining dimension reduction and neural network training. The BPVM and KPVM are called projection vector machine (PVM) by a joint name. Different from random weight idea used in ELM, in PVM, the input weights of SLFN are obtained by singular decomposition, and the hidden neurons are ranked by their contribution level and only those that are important are selected. This will make PVM to obtain better generalization ability and more compact structure than ELM. Similar to ELM, PVM does not need any iterative steps, and thus can yield significantly fast training. Additionally, compared with the two-stage algorithms such as BP/SVD (BP co-work with singular value decomposition) and ELM/SVD (ELM co-work with singular value decomposition), PVM need less parameters and memory running space. The experimental results on many datasets show that; although PVM is not faster than ELM, it is much faster than BP, BP/SVD and ELM/SVD which produce the best generalization ability in most cases. Especially, PVM is very suitable for high-dimension and small-sample dataset.

Incremental Projection Vector Machine: A One-Stage Learning Algorithm for High-Dimension Large-Sample Dataset

Projection Vector Machine: One-Stage Learning Algorithm From High-Dimension Small-Sample Data

Incremental Learning Algorithm Based on Support Vector Machine

An Incremental Updating Method for Support Vector Machines

An Incremental Learning Algorithm Based on Support Vector Domain Classifier

AnIncremental Learning Algorithm Basedon Support Vector DomainClassifier

Incremental batch learning with support vector machines

Fast incremental learning algorithm of linear support vector machine based on hull vectors

Projection Vector Machine.

An Online Incremental Learning Support Vector Machine for Large-Scale Data

Inverse Matrix-Free Incremental Proximal Support Vector Machine

Local Feature Discriminant Projection

Fast Incremental Learning Method for One-Class Support Vector Machine

Incremental Least Squares Classifier Vs.incremental Proximal Support Vector Machine

Approximate Approach to Train SVM on Very Large Data Sets

Incremental Learning Algorithm for Support Vector Data Description

IDR/QR: an Incremental Dimension Reduction Algorithm Via QR Decomposition

A novel online incremental and decremental learning algorithm based on variable support vector machine

Multi-Sample Incremental Manifold Learning Algorithm Based on Isogonal Mapping

A Gradual Training Algorithm of Incremental Support Vector Machine Learning