Abstract:The random weight idea in neural network has existed for long time and attracted wide attention for its faster learning speed than that of tuning-based methods. Recently, extreme learning machine (ELM), one non-tuning algorithm for single hidden layer neural network (SLFN) was proposed and applied in many applications. ELM can learn hundreds of times faster than that of the traditional tuning-based methods while obtaining better generalization performance. But since the ELM randomly assigns input weights, its stability and accuracy will decline sharply for high-dimension and small-sample dataset. Although dimension reduction can be used to improve it, this two-stage method will need more physical space and increase structure complexity of the model. In this paper, we propose a novel single-stage algorithm, base projection vector machine (BPVM) and its kernel version—kernel projection vector machine (KPVM) by combining dimension reduction and neural network training. The BPVM and KPVM are called projection vector machine (PVM) by a joint name. Different from random weight idea used in ELM, in PVM, the input weights of SLFN are obtained by singular decomposition, and the hidden neurons are ranked by their contribution level and only those that are important are selected. This will make PVM to obtain better generalization ability and more compact structure than ELM. Similar to ELM, PVM does not need any iterative steps, and thus can yield significantly fast training. Additionally, compared with the two-stage algorithms such as BP/SVD (BP co-work with singular value decomposition) and ELM/SVD (ELM co-work with singular value decomposition), PVM need less parameters and memory running space. The experimental results on many datasets show that; although PVM is not faster than ELM, it is much faster than BP, BP/SVD and ELM/SVD which produce the best generalization ability in most cases. Especially, PVM is very suitable for high-dimension and small-sample dataset.

A Projection Neural Network for Training Support Vector Machines

Flatheaded support vector machine for regression

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Sparse Representation Based on Projection Method in Online Least Squares Support Vector Machines

A modified projection neural network for linear variational inequalities and quadratic optimization problems

Subgradient-Based Neural Network for Nonconvex Optimization Problems in Support Vector Machines with Indefinite Kernels

A Project Neural Network for Solving Degenerate Convex Quadratic Program

Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network.

Projection Vector Machine: One-Stage Learning Algorithm From High-Dimension Small-Sample Data

A Nonlinear Projection Neural Network for Solving Interval Quadratic Programming Problems and Its Stability Analysis

Learnable Subspace Orthogonal Projection for Semi-supervised Image Classification

A Novel Projection Neural Network for Solving a Class of Monotone Variational Inequalities.

A New Delayed Projection Neural Network for Solving Linear Variational Inequalities and Quadratic Optimization Problems

A sparse semismooth Newton based augmented Lagrangian method for large-scale support vector machines

Design of General Projection Neural Networks for Solving Monotone Linear Variational Inequalities and Linear and Quadratic Optimization Problems

A Novel Neural Network for Nonlinear Convex Programming

A compact neural network for training support vector machines.

A Recurrent Neural Network with Finite-Time Convergence for Convex Quadratic Bilevel Programming Problems

A Neutral-Type Delayed Projection Neural Network for Solving Quadratic Programming Problems

A Project Neural Network for Solving Degenerate Quadratic Minimax Problem with Linear Constraints

Projection Vector Machine.