Abstract:The backpropagation (BP) neural network has been widely used in many fields. However, it is still a great challenge to design the architecture and obtain optimal parameters for BP neural networks. For improving the generalization performance, regularization is the most popular technique to train the BP neural networks. In this paper, we propose a novel BP algorithm with graph regularization (BPGR) to obtain optimal parameters, by imposing the graph regularization term to the error function. The essential idea is to force the latent features of hidden layer to be more concentrated, which enhances the generalization performance. Besides, the proposed modified graph regularization facilitates the calculation of gradient and is more capable to penalize the extreme values of weights. Furthermore, the graph regularization can also be integrated with deep neural networks to improve their generalization performance. In addition, we provide the convergence analysis of our method BPGR under some regularity conditions. By comparison on several datasets with five activation functions, experimental results validate the theoretical analysis and demonstrate outstanding performance of BPGR.

A Modified Algorithm for Feedforward Neural Networks

Improved BP algorithm for neural network and its application in welding process control

A Learning Algorithm for Solving the Over-Fitting Problem in BPANN

A New Global Optimization Algorithm for Training Feedforward Neural Networks and Its Application

BP algorithm based on network complexity

The Improvements of Bp Neural Network Learning Algorithm

An Improved Back-propagation NN Algorithm And Its Application

Optimal parameters selection of back propagation algorithm in the feedforward neural network

New Learning Algorithm for Feed-Forward Neural Network Based on Objective Back-Propagation

Research of New Learning Method of Feedforward Neural Network

An Effective Backpropagation Algorithm for Optimizing BP Neural Network Based on Rough Set and Modified Genetic Algorithm

A Modified Learning Algorithm for Improving the Fault Tolerance of BP Networks

Brief Introduction of Back Propagation (BP) Neural Network Algorithm and Its Improvement

Study on Improved FIR Filters Using BP Neural Networks

A BP Neural Networks Learning Algorithm Research based on Breeding Algorithm

A Learning Algorithm for Feedforward Neural Networks Based on Fuzzy Controller

A Fast Learning Strategy For Multilayer Feedforward Neural Networks

Fast Learning Algorithms for Feedforward Neural Networks

A Backpropagation Learning Algorithm with Graph Regularization for Feedforward Neural Networks

An Improved Model of BP Neural Network Algorithm

Multilayered Feed Forward Neural Network Based on Particle Swarm Optimizer Algorithm