Abstract:The slow convergence of back-propagation neural network (BPNN) has become a challenge in data-mining and knowledge discovery applications due to the drawbacks of the gradient descent (GD) optimization method, which is widely adopted in BPNN learning. To solve this problem, some standard optimization techniques such as conjugate-gradient and Newton method have been proposed to improve the convergence rate of BP learning algorithm. This paper presents a heuristic method that adds an adaptive smoothing momentum term to original BP learning algorithm to speedup the convergence. In this improved BP learning algorithm, adaptive smoothing technique is used to adjust the momentums of weight updating formula automatically in terms of “3 σ limits theory.” Using the adaptive smoothing momentum terms, the improved BP learning algorithm can make the network training and convergence process faster, and the network’s generalization performance stronger than the standard BP learning algorithm can do. In order to verify the effectiveness of the proposed BP learning algorithm, three typical foreign exchange rates, British pound (GBP), Euro (EUR), and Japanese yen (JPY), are chosen as the forecasting targets for illustration purpose. Experimental results from homogeneous algorithm comparisons reveal that the proposed BP learning algorithm outperforms the other comparable BP algorithms in performance and convergence rate. Furthermore, empirical results from heterogeneous model comparisons also show the effectiveness of the proposed BP learning algorithm.

A Back-Propagation Algorithm with Adaptive Momentum Factor

A New BP Algorithm with Adaptive Momentum for FNNs Training

Convergence of Gradient Method with Momentum for Back-Propagation Neural Networks

Convergence analysis of a back-propagation algorithm with adaptive momentum

Research on RBM Accelerating Learning Algorithm with Weight Momentum

The Improvements of Bp Neural Network Learning Algorithm

A synthetically backpropagation algorithm

Back Propagation Algorithm Using Dynamic Optimal Learning Rate

An Algorithm for Fast Convergence of Back Propagation by Enlarging Error

The Implementation of the Improved BP Algorithm by Adding the Item of the Momentum

The Performance of the Backpropagation Algorithm with Varying Slope of the Activation Function

Modified Blind Equalization Algorithm Based on Back-Propagation Neural Networks with Adaptive Momentum Factor

An Improved Back-propagation NN Algorithm And Its Application

Adaptive Back-Propagation Algorithm with Magnified Error Signals

Forecasting foreign exchange rates with an improved back-propagation learning algorithm with adaptive smoothing momentum terms

Boundedness and Convergence of Split-Complex Back-Propagation Algorithm with Momentum and Penalty

Convergence of a Batch Gradient Algorithm with Adaptive Momentum for Neural Networks

An Improved Back-Propagation Algorithm Applied To Target Recognition

A New Adaptive Momentum Algorithm for Split-Complex Recurrent Neural Networks

Back Propagation Based on Selective Attention for Fast Convergence of Training Neural Network

Joint-optimization algorithm of BP neural network