Convergence of A Gradient Algorithm with Penalty for Training Two-Layer Neural Networks

Hongmei Shao,Lijun Liu,Gaofeng Zheng
DOI: https://doi.org/10.1109/iccsit.2009.5234616
2009-01-01
Abstract:In this paper a squared penalty term is added to the conventional error function to improve the generalization of neural networks. A weight boundedness theorem and two convergence theorems are proved for the gradient learning algorithm with penalty when it is used for training a two-layer feedforward neural network. To illustrate above theoretical findings, numerical experiments are conducted based on a linearly separable problem and simulation results are presented. The abstract goes here.
What problem does this paper attempt to address?