Abstract:Subspace minimization conjugate gradient (SMCG) methods are a class of quite efficient iterative methods for unconstrained optimization and have received increasing attention recently. The search directions of SMCG methods are generated by minimizing an approximate model with the approximate matrix over the two-dimensional subspace spanned by the current gradient and the latest step. The main drawback of SMCG methods is that the parameter in the search directions must be determined when calculating the search directions. The parameter is crucial to SMCG methods and is difficult to be determined properly. An alternative solution for this drawback might be to exploit a new way to derive SMCG methods independent of . The projection technique has been used successfully to derive conjugate gradient directions such as the Dai–Kou conjugate gradient direction (Dai and Kou in SIAM J Optim 23(1):296–320, 2013). Motivated by the above two observations, we use a projection technique to derive a new SMCG method independent of . More specifically, we project the search direction of memoryless quasi-Newton method into the above two-dimensional subspace and derive a new search direction, which is proved to be descent. Remarkably, the proposed method without any line search enjoys the finite termination property for two-dimensional strictly convex quadratic functions. An adaptive scaling factor in the search direction is exploited based on the finite termination property. The proposed method does not need to determine the parameter and can be regarded as an extension of the Dai–Kou conjugate gradient method. The global convergence of the proposed method is established under the suitable assumptions. Numerical comparisons on the 147 test functions from the CUTEst library indicate that the proposed method is very promising.

Several Accelerated Subspace Minimization Conjugate Gradient Methods Based on Regularization Model and Convergence Rate Analysis for Nonconvex Problems.

The convergence of conjugate gradient method with nonmonotone line search

A New Subspace Minimization Conjugate Gradient Method for Unconstrained Minimization

A regularized limited memory subspace minimization conjugate gradient method for unconstrained optimization

An efficient inertial subspace minimization CG algorithm with convergence rate analysis for constrained nonlinear monotone equations

Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

A family of three-term conjugate gradient projection methods with a restart procedure and their relaxed-inertial extensions for the constrained nonlinear pseudo-monotone equations with applications

Two efficient gradient methods with approximately optimal stepsizes based on regularization models for unconstrained optimization

Regularized Step Directions in Nonlinear Conjugate Gradient Methods

A class of new three-term descent conjugate gradient algorithms for large-scale unconstrained optimization and applications to image restoration problems

Some generalized three-term conjugate gradient methods based on CD approach for unconstrained optimization problems

A family of limited memory three term conjugate gradient methods

A limited memory subspace minimization conjugate gradient algorithm for unconstrained optimization

On the Global Convergence of Majorization Minimization Algorithms for Nonconvex Optimization Problems.

Quadratically Regularized Subgradient Methods for Weakly Convex Optimization with Weakly Convex Constraints

An Accelerated Stochastic ADMM for Nonconvex and Nonsmooth Finite-Sum Optimization

A Momentum Accelerated Adaptive Cubic Regularization Method for Nonconvex Optimization

Inexact Proximal Cubic Regularized Newton Methods for Convex Optimization

Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints