Abstract:Subspace minimization conjugate gradient (SMCG) methods are a class of quite efficient iterative methods for unconstrained optimization. The orthogonality is an important property of linear conjugate gradient method. It is however observed that the orthogonality of the gradients in linear conjugate gradient method is often lost, which usually causes slow convergence. Based on SMCG BB (Liu and Liu in J Optim Theory Appl 180(3):879–906, 2019), we combine subspace minimization conjugate gradient method with the limited memory technique and present a limited memory subspace minimization conjugate gradient algorithm for unconstrained optimization. The proposed method includes two types of iterations: SMCG iteration and quasi-Newton (QN) iteration. In the SMCG iteration, the search direction is determined by solving a quadratic approximation problem, in which the important parameter is estimated based on some properties of the objective function at the current iterative point. In the QN iteration, a modified quasi-Newton method in the subspace is proposed to improve the orthogonality. Additionally, a modified strategy for choosing the initial stepsize is exploited. The global convergence of the proposed method is established under weak conditions. Some numerical results indicate that, for the tested functions in the CUTEr library, the proposed method has a great improvement over SMCG BB, and it is comparable to the latest limited memory conjugate gradient software package CG DESCENT (6.8) (Hager and Zhang in SIAM J Optim 23(4):2150–2168, 2013) and is also superior to the famous limited memory BFGS (L-BFGS) method.

An Improved Dai–Kou Conjugate Gradient Algorithm for Unconstrained Optimization

The convergence of conjugate gradient method with nonmonotone line search

A New Dai-Liao Conjugate Gradient Method based on Approximately Optimal Stepsize for Unconstrained Optimization

Two efficient gradient methods with approximately optimal stepsizes based on regularization models for unconstrained optimization

An Improved Gradient Method with Approximately Optimal Stepsize Based on Conic model for Unconstrained Optimization

An improved Dai‐Liao‐style hybrid conjugate gradient‐based method for solving unconstrained nonconvex optimization and extension to constrained nonlinear monotone equations

A revised Liu–Storey conjugate gradient parameter for unconstrained optimization problems with applications

A New Hybrid Three-Term LS-CD Conjugate Gradient In Solving Unconstrained Optimization Problems

Modified parameter of Dai Liao conjugacy condition of the conjugate gradient method

A Modified Dai–Liao Conjugate Gradient Method Based on a Scalar Matrix Approximation of Hessian and Its Application

A decent three term conjugate gradient method with global convergence properties for large scale unconstrained optimization problems

A three-term conjugate gradient descent method with some applications

Proposed CG method to solve unconstrained optimization problems

A class of new three-term descent conjugate gradient algorithms for large-scale unconstrained optimization and applications to image restoration problems

A limited memory subspace minimization conjugate gradient algorithm for unconstrained optimization

A self-scaling memoryless BFGS based conjugate gradient method using multi-step secant condition for unconstrained minimization

Two modified conjugate gradient methods for unconstrained optimization

A Descent Conjugate Gradient Algorithm for Optimization Problems and Its Applications in Image Restoration and Compression Sensing

A New Hybrid Descent Algorithm for Large-Scale Nonconvex Optimization and Application to Some Image Restoration Problems

An efficient inertial subspace minimization CG algorithm with convergence rate analysis for constrained nonlinear monotone equations

A New Scaled Three-Term Conjugate Gradient Algorithms For Unconstrained Optimization