Abstract:Subspace minimization conjugate gradient (SMCG) methods are a class of quite efficient iterative methods for unconstrained optimization. The orthogonality is an important property of linear conjugate gradient method. It is however observed that the orthogonality of the gradients in linear conjugate gradient method is often lost, which usually causes slow convergence. Based on SMCG BB (Liu and Liu in J Optim Theory Appl 180(3):879–906, 2019), we combine subspace minimization conjugate gradient method with the limited memory technique and present a limited memory subspace minimization conjugate gradient algorithm for unconstrained optimization. The proposed method includes two types of iterations: SMCG iteration and quasi-Newton (QN) iteration. In the SMCG iteration, the search direction is determined by solving a quadratic approximation problem, in which the important parameter is estimated based on some properties of the objective function at the current iterative point. In the QN iteration, a modified quasi-Newton method in the subspace is proposed to improve the orthogonality. Additionally, a modified strategy for choosing the initial stepsize is exploited. The global convergence of the proposed method is established under weak conditions. Some numerical results indicate that, for the tested functions in the CUTEr library, the proposed method has a great improvement over SMCG BB, and it is comparable to the latest limited memory conjugate gradient software package CG DESCENT (6.8) (Hager and Zhang in SIAM J Optim 23(4):2150–2168, 2013) and is also superior to the famous limited memory BFGS (L-BFGS) method.

A revised Liu–Storey conjugate gradient parameter for unconstrained optimization problems with applications

The convergence of conjugate gradient method with nonmonotone line search

A New Hybrid Three-Term LS-CD Conjugate Gradient In Solving Unconstrained Optimization Problems

Two sufficient descent spectral conjugate gradient algorithms for unconstrained optimization with application

Modified parameter of Dai Liao conjugacy condition of the conjugate gradient method

Proposed CG method to solve unconstrained optimization problems

A three-term conjugate gradient descent method with some applications

A decent three term conjugate gradient method with global convergence properties for large scale unconstrained optimization problems

A new family of hybrid three-term conjugate gradient method for unconstrained optimization with application to image restoration and portfolio selection

A New Scaled Three-Term Conjugate Gradient Algorithms For Unconstrained Optimization

An efficient inertial subspace minimization CG algorithm with convergence rate analysis for constrained nonlinear monotone equations

A limited memory subspace minimization conjugate gradient algorithm for unconstrained optimization

A Conjugate Gradient Method with Global Convergence for Large-Scale Unconstrained Optimization Problems

A new parameter in three-term conjugate gradient algorithms for unconstrained optimization

A Modified Conjugacy Condition And Related Nonlinear Conjugate Gradient Method

A family of limited memory three term conjugate gradient methods

Spectral conjugate gradient methods for vector optimization problems

The conjugate gradient method with various viewpoints

A HYBRID CONJUGATE GRADIENT METHOD FOR UNCONSTRAINED OPTIMIZATION WITH APPLICATION

A self-scaling memoryless BFGS based conjugate gradient method using multi-step secant condition for unconstrained minimization

A scaled three-term conjugate gradient method for unconstrained optimization