Abstract:The stationary point of optimization problems can be obtained via conjugate gradient (CG) methods without the second derivative. Many researchers have used this method to solve applications in various fields, such as neural networks and image restoration. In this study, we construct a three-term CG method that fulfills convergence analysis and a descent property. Next, in the second term, we employ a Hestenses-Stiefel CG formula with some restrictions to be positive. The third term includes a negative gradient used as a search direction multiplied by an accelerating expression. We also provide some numerical results collected using a strong Wolfe line search with different sigma values over 166 optimization functions from the CUTEr library. The result shows the proposed approach is far more efficient than alternative prevalent CG methods regarding central processing unit (CPU) time, number of iterations, number of function evaluations, and gradient evaluations. Moreover, we present some applications for the proposed three-term search direction in image restoration, and we compare the results with well-known CG methods with respect to the number of iterations, CPU time, as well as root-mean-square error (RMSE). Finally, we present three applications in regression analysis, image restoration, and electrical engineering.

Conjugate Gradient Algorithm Based on Directions of Negative Curvature in Complex Domain

Exploiting Negative Curvature in Conjunction with Adaptive Sampling: Theoretical Results and a Practical Algorithm

A hybrid inexact regularized Newton and negative curvature method

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Saving Gradient and Negative Curvature Computations: Finding Local Minima More Efficiently

A New Hybrid Descent Algorithm for Large-Scale Nonconvex Optimization and Application to Some Image Restoration Problems

A Convex Combination between Two Different Search Directions of Conjugate Gradient Method and Application in Image Restoration

A Scaled Conjugate Gradient Method with Moving Asymptotes for Unconstrained Optimization Problems

A Descent Conjugate Gradient Algorithm for Optimization Problems and Its Applications in Image Restoration and Compression Sensing

Some three-term conjugate gradient methods with the new direction structure

A three-term conjugate gradient descent method with some applications

A Modified Nonlinear Conjugate Gradient Algorithm for Large-Scale Nonsmooth Convex Optimization

Applications of the Conjugate Gradient Method in Optimal Surface Parameterizations

Application of gradient descent algorithms based on geodesic distances

A Revised Conjugate Gradient Projection Algorithm for Inequality Constrained Optimizations

A Conjugate Gradient Method with Descent Direction for Unconstrained Optimization

Quantum algorithm for finding the negative curvature direction in non-convex optimization

A Modified Dai–Liao Conjugate Gradient Method Based on a Scalar Matrix Approximation of Hessian and Its Application

The conjugate gradient method with various viewpoints

Online Learning Guided Curvature Approximation: A Quasi-Newton Method with Global Non-Asymptotic Superlinear Convergence

An inertial Dai-Liao conjugate method for convex constrained monotone equations that avoids the direction of maximum magnification