Abstract:The backpropagation method of neural network (BPNN) method which is an important algorithm in machine learning has been applied to wide range of problem like pattern recognition, optimization, approximation, classification, and data clustering in real world. BPNN algorithm has been widely used in age estimation, pedestrian gender classification, traffic sign recognition, character recognition, water pollution forecasting models, heart disease classification, breast cancer detection (Keskar et al. in On large-batch training for deep learning: Generalization gap and sharp minima, 2016) [1], remote sensing, and image classification (He et al. in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 558–567, 2019) [2]. Algorithm uses a steepest gradient method and suffers with some limitations like convergence to local minima and slow convergence velocity of learning. This research proposed solution for the slow learning convergence velocity by implementing learning rate annealing which implements anneal the learning rate (decline as time progresses) rather than constant learning rate throughout the training. The problem of local minima can be address by momentum, which can be calculated by adding fraction of the past weights updates to the calculation of current weight.

Research of New Learning Method of Feedforward Neural Network

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

Fast Learning Algorithms for Feedforward Neural Networks

Learning biped locomotion based on Q-learning and neural networks

Fast Algorithm for Adaptive Generalized Predictive Control Based on Bp Neural Networks

A Nonlinear Adaptive Controller Based on RBF Networks

Brief Introduction of Back Propagation (BP) Neural Network Algorithm and Its Improvement

A Fast Learning Algorithm of Neural Networks by Changing Error Functions

Optimal parameters selection of back propagation algorithm in the feedforward neural network

A Fast Hybrid Algorithm Of Global Optimization For Feedforward Neural Networks

New and Rapidly Converging BP Neural Network Algorithm

Two Hybrid Learning Strategies of Feedforward Network

Feedforward neural networks and training methods

A PSO-BP Model for Nonlinear Fitting of Phyllostachys edulis' Thermal Conductivity

Research and application on improved BP neural network algorithm

Feed-Forward Optimization With Delayed Feedback for Neural Networks

Improving Steepest Descent Method by Learning Rate Annealing and Momentum in Neural Network

Memorized Sparse Backpropagation.

A Fast Learning Algorithm of Feedforward Neural Networks by Using Novel Error Functions

A constrained optimization method based on BP neural network

A Fast Learning Algorithm Of Neural Network For The Training And Recognition Of The Phonemes