Abstract:Backpropagation neural networks are commonly utilized to solve complicated issues in various disciplines. However, optimizing their settings remains a significant task. Traditional gradient-based optimization methods, such as stochastic gradient descent (SGD), often exhibit slow convergence and hyperparameter sensitivity. An adaptive stochastic conjugate gradient (ASCG) optimization strategy for backpropagation neural networks is proposed in this study. ASCG combines the advantages of stochastic optimization and conjugate gradient techniques to increase training efficiency and convergence speed. Based on the observed gradients, the algorithm adaptively calculates the learning rate and search direction at each iteration, allowing for quicker convergence and greater generalization. Experimental findings on benchmark datasets show that ASCG optimization outperforms standard optimization techniques regarding convergence time and model performance. The proposed ASCG algorithm provides a viable method for improving the training process of backpropagation neural networks, making them more successful in tackling complicated problems across several domains. As a result, the information for initial seeds formed while the model is being trained grows. The coordinated efforts of ASCG's Conjugate Gradient and ASCG components improve learning and achieve global minima. Our results indicate that our ASCG algorithm achieves 21 percent higher accuracy on the HMT dataset and performs better than existing methods on other datasets(DIR-Lab dataset). The experimentation revealed that the conjugate gradient has an efficiency of 95 percent when utilizing the principal component analysis features, compared to 94 percent when using the correlation heatmap features selection approach with MSE of 0.0678.

Training Feed-forward Neural Networks Using the Gradient Descent Method with the Optimal Stepsize

Optimal parameters selection of back propagation algorithm in the feedforward neural network

Fast Learning Algorithms for Feedforward Neural Networks

An Algorithm for Fast Convergence of Back Propagation by Enlarging Error

A hybrid training algorithm based on gradient descent and evolutionary computation

The Study of Approximate Optimal Step Size of B-P Neural Network

A New Global Optimization Algorithm for Training Feedforward Neural Networks and Its Application

Adaptive Stochastic Conjugate Gradient Optimization for Backpropagation Neural Networks

Step-size Optimization for Continual Learning

Self-adaptation Optimize BP Neural Network Design Based on the Genetic Algorithms

Learning the Step-size Policy for the Limited-Memory Broyden-Fletcher-Goldfarb-Shanno Algorithm

A Global Optimization Algorithm For Training Feedforward Neural Networks

Back Propagation Based on Selective Attention for Fast Convergence of Training Neural Network

Optimal Algorithm for a Multilayer Feedforward Net and Its Application

Semi-Implicit Back Propagation.

Improving Steepest Descent Method by Learning Rate Annealing and Momentum in Neural Network

Gradient Descent: The Ultimate Optimizer

Optimization Study of BP Neural Network Based on Genetic Algorithm

Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization

A Fast Learning Algorithm Of Neural Network For The Training And Recognition Of The Phonemes

Global Convergence of Online Bp Training with Dynamic Learning Rate