Abstract:Backpropagation neural networks are commonly utilized to solve complicated issues in various disciplines. However, optimizing their settings remains a significant task. Traditional gradient-based optimization methods, such as stochastic gradient descent (SGD), often exhibit slow convergence and hyperparameter sensitivity. An adaptive stochastic conjugate gradient (ASCG) optimization strategy for backpropagation neural networks is proposed in this study. ASCG combines the advantages of stochastic optimization and conjugate gradient techniques to increase training efficiency and convergence speed. Based on the observed gradients, the algorithm adaptively calculates the learning rate and search direction at each iteration, allowing for quicker convergence and greater generalization. Experimental findings on benchmark datasets show that ASCG optimization outperforms standard optimization techniques regarding convergence time and model performance. The proposed ASCG algorithm provides a viable method for improving the training process of backpropagation neural networks, making them more successful in tackling complicated problems across several domains. As a result, the information for initial seeds formed while the model is being trained grows. The coordinated efforts of ASCG's Conjugate Gradient and ASCG components improve learning and achieve global minima. Our results indicate that our ASCG algorithm achieves 21 percent higher accuracy on the HMT dataset and performs better than existing methods on other datasets(DIR-Lab dataset). The experimentation revealed that the conjugate gradient has an efficiency of 95 percent when utilizing the principal component analysis features, compared to 94 percent when using the correlation heatmap features selection approach with MSE of 0.0678.

An Improved Conjugate Gradient Neural Networks Based on a Generalized Armijo Search Method

The convergence of conjugate gradient method with nonmonotone line search

Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and Bp Algorithms

Back-Propagation Network Improved by Conjugate Gradient Based on Genetic Algorithm in Qsar Study on Endocrine Disrupting Chemicals

Emergency response to the Haitian earthquake--as bad as it gets.

An optimizing BP neural network algorithm based on genetic algorithm

Adaptive Stochastic Conjugate Gradient Optimization for Backpropagation Neural Networks

A three-term conjugate gradient descent method with some applications

Optimization Study of BP Neural Network Based on Genetic Algorithm

A hybrid training algorithm based on gradient descent and evolutionary computation

A decent three term conjugate gradient method with global convergence properties for large scale unconstrained optimization problems

An Improved Adam Optimization Algorithm Combining Adaptive Coefficients and Composite Gradients Based on Randomized Block Coordinate Descent

A New Hybrid Three-Term LS-CD Conjugate Gradient In Solving Unconstrained Optimization Problems

Brief Introduction of Back Propagation (BP) Neural Network Algorithm and Its Improvement

GACNN: Training Deep Convolutional Neural Networks with Genetic Algorithm

A Fast Hybrid Algorithm Of Global Optimization For Feedforward Neural Networks

Back Propagation With Randomized Cost Function For Training Neural Networks

An Improved Three-Term Conjugate Gradient Algorithm for Constrained Nonlinear Equations under Non-Lipschitz Conditions and Its Applications

Gradient Correction Beyond Gradient Descent

The Improved Rapid Convergence Algorithm of the Connecting Rights in the BP Network

Some generalized three-term conjugate gradient methods based on CD approach for unconstrained optimization problems