Abstract:Backpropagation neural networks are commonly utilized to solve complicated issues in various disciplines. However, optimizing their settings remains a significant task. Traditional gradient-based optimization methods, such as stochastic gradient descent (SGD), often exhibit slow convergence and hyperparameter sensitivity. An adaptive stochastic conjugate gradient (ASCG) optimization strategy for backpropagation neural networks is proposed in this study. ASCG combines the advantages of stochastic optimization and conjugate gradient techniques to increase training efficiency and convergence speed. Based on the observed gradients, the algorithm adaptively calculates the learning rate and search direction at each iteration, allowing for quicker convergence and greater generalization. Experimental findings on benchmark datasets show that ASCG optimization outperforms standard optimization techniques regarding convergence time and model performance. The proposed ASCG algorithm provides a viable method for improving the training process of backpropagation neural networks, making them more successful in tackling complicated problems across several domains. As a result, the information for initial seeds formed while the model is being trained grows. The coordinated efforts of ASCG's Conjugate Gradient and ASCG components improve learning and achieve global minima. Our results indicate that our ASCG algorithm achieves 21 percent higher accuracy on the HMT dataset and performs better than existing methods on other datasets(DIR-Lab dataset). The experimentation revealed that the conjugate gradient has an efficiency of 95 percent when utilizing the principal component analysis features, compared to 94 percent when using the correlation heatmap features selection approach with MSE of 0.0678.

The Conjugate Gradient Method with neural network control

The convergence of conjugate gradient method with nonmonotone line search

A Novel Conjugate Gradient Method with Generalized Armijo Search for Efficient Training of Feedforward Neural Networks

New Conjugate Gradient-Like Methods For Unconstrained Optimization

New Conjugacy Condition and Related New Conjugate Gradient Methods for Unconstrained Optimization

Adaptive Stochastic Conjugate Gradient Optimization for Backpropagation Neural Networks

A three-term conjugate gradient descent method with some applications

A Conjugate Gradient Method with Global Convergence for Large-Scale Unconstrained Optimization Problems

A Modified Conjugate Gradient-Based Elman Neural Network

An Improved Dai–Kou Conjugate Gradient Algorithm for Unconstrained Optimization

An Improved Nonlinear Conjugate Gradient Method with an Optimal Property

Variable three-term conjugate gradient method for training artificial neural networks

An Improved Conjugate Gradient Neural Networks Based on a Generalized Armijo Search Method

A Neural Network Transformation based Global Optimization Algorithm

A Modified Conjugacy Condition And Related Nonlinear Conjugate Gradient Method

Proposed CG method to solve unconstrained optimization problems

A Conjugate Gradient Algorithm with Function Value Information and N-Step Quadratic Convergence for Unconstrained Optimization.

Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks

Fully Complex Conjugate Gradient-Based Neural Networks Using Wirtinger Calculus Framework: Deterministic Convergence and Its Application.

Stochastic three-term conjugate gradient method with variance technique for non-convex learning

Concaving Space Algorithm For Neural Network