Abstract:A back propagation neural network based on selective attention (SABP) is proposed for improving the learning speed of multi-layer artificial neural networks with sigmoid activation function. The algorithm embedded the selective attention model in the NN training. The key to this algorithm lies in the pre-attention function, in which primitive features of the stimulus were detected in parallel by what might be called feature detectors by an improved GA, and the attention function, in which detail features were handled with the attention spotlight by the BP with an enlarged error item in the output layer. Therefore, the SABP obtained the ability of searching the global optimum solution relying on the pre-attention function that made a lower fault rate to drop into local optimum, and the attention function updated the weights of neural network effectively by enlarging the error in the output layer, which kept a high leaning rate to meet the convergence criteria quickly. The simulation experiments demonstrate that the algorithm is more powerful than Momentum algorithm or Delta-bar-Delta rule in learning, and it can effectively avoid failure training caused by randomizing the initial weights and thresholds, and solve the slow convergence problem resulting from the Flat-Spots when the error signal becomes too small.

Fast Learning Algorithms for Feedforward Neural Networks

A Fast Learning Algorithm Of Neural Network For The Training And Recognition Of The Phonemes

A Fast Hybrid Algorithm Of Global Optimization For Feedforward Neural Networks

A Fast Learning Algorithm of Neural Networks by Changing Error Functions

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

A Fast Learning Strategy For Multilayer Feedforward Neural Networks

A Fast Learning Algorithm of Feedforward Neural Networks by Using Novel Error Functions

A Fast Learning Algorithm for Feedfoward Neural Network Based on the Layer-by-Layer and Neuron-by-Neuron Optimizing Procedure

Optimal Algorithm for a Multilayer Feedforward Net and Its Application

Feedback control standpoint based fast training approach of neural networks and its application

Training Feed-forward Neural Networks Using the Gradient Descent Method with the Optimal Stepsize

Optimal filtering algorithms for fast learning in feedforward neural networks

An Algorithm for Fast Convergence of Back Propagation by Enlarging Error

Speeding Up the Training Process of the Mfnn by Optimizing the Hidden Layers' Outputs

A New BP Algorithm with Adaptive Momentum for FNNs Training

Optimal parameters selection of back propagation algorithm in the feedforward neural network

Maximization of the Gradient Function for Efficient Neural Network Training

The Improvements of Bp Neural Network Learning Algorithm

Back Propagation Based on Selective Attention for Fast Convergence of Training Neural Network

Research of New Learning Method of Feedforward Neural Network

A New Layer-Wise Linearized Algorithm for Feedforward Neural Network