Abstract:A back propagation neural network based on selective attention (SABP) is proposed for improving the learning speed of multi-layer artificial neural networks with sigmoid activation function. The algorithm embedded the selective attention model in the NN training. The key to this algorithm lies in the pre-attention function, in which primitive features of the stimulus were detected in parallel by what might be called feature detectors by an improved GA, and the attention function, in which detail features were handled with the attention spotlight by the BP with an enlarged error item in the output layer. Therefore, the SABP obtained the ability of searching the global optimum solution relying on the pre-attention function that made a lower fault rate to drop into local optimum, and the attention function updated the weights of neural network effectively by enlarging the error in the output layer, which kept a high leaning rate to meet the convergence criteria quickly. The simulation experiments demonstrate that the algorithm is more powerful than Momentum algorithm or Delta-bar-Delta rule in learning, and it can effectively avoid failure training caused by randomizing the initial weights and thresholds, and solve the slow convergence problem resulting from the Flat-Spots when the error signal becomes too small.

A Fast Learning Algorithm of Neural Networks by Changing Error Functions

Fast Learning Algorithms for Feedforward Neural Networks

A Fast Learning Algorithm of Feedforward Neural Networks by Using Novel Error Functions

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

An Algorithm for Fast Convergence of Back Propagation by Enlarging Error

A Fast Learning Strategy For Multilayer Feedforward Neural Networks

The Improvements of Bp Neural Network Learning Algorithm

A Fast Learning Algorithm Of Neural Network For The Training And Recognition Of The Phonemes

Feedback control standpoint based fast training approach of neural networks and its application

Optimal Algorithm for a Multilayer Feedforward Net and Its Application

A Fast Learning Algorithm for Feedfoward Neural Network Based on the Layer-by-Layer and Neuron-by-Neuron Optimizing Procedure

Maximization of the Gradient Function for Efficient Neural Network Training

Optimal parameters selection of back propagation algorithm in the feedforward neural network

A New Layer-Wise Linearized Algorithm for Feedforward Neural Network

Speeding Up the Training Process of the Mfnn by Optimizing the Hidden Layers' Outputs

Brief Introduction of Back Propagation (BP) Neural Network Algorithm and Its Improvement

New Learning Algorithm for Feed-Forward Neural Network Based on Objective Back-Propagation

The Improved Rapid Convergence Algorithm of the Connecting Rights in the BP Network

An Improved Back-propagation NN Algorithm And Its Application

A New BP Algorithm with Adaptive Momentum for FNNs Training

Back Propagation Based on Selective Attention for Fast Convergence of Training Neural Network