Abstract:Quasi-Newton methods form an important class of methods for solving nonlinear optimization problems. In such methods, first order information is used to approximate the second derivative. The aim is to mimic the fast convergence that can be guaranteed by Newton-based methods. In the best case, quasi-Newton methods will far outperform steepest descent and other first order methods, without the computational cost of calculating the exact second derivative. These convergence guarantees hold locally, which follows closely from the fact that, if the objective function is strongly convex, it can be approximated well by a quadratic function close to the solution. Understanding the performance of quasi-Newton methods on quadratic problems with a symmetric positive definite Hessian is therefore of vital importance. In the classic case, an approximation of the Hessian is updated at every iteration and exact line search is used. It is well known that the algorithm terminates finitely, even when the Hessian approximation is memoryless, i.e. requires only the most recent information. In this paper, we explore the possibilities in which reliance on exact line search and dependence on conjugate search directions can be relaxed, while preserving finite termination properties of quasi-Newton methods on quadratic problems. We show that it suffices to create a memoryless quasi-Newton matrix based on two vectors to give ability to compute a Newton direction within a finite number of iterations, independent of step lengths. It is unnecessary for the quasi-Newton approximation to act as the Hessian on the full space.

Comments on finite termination of the generalized Newton method for absolute value equations

Comments on finite termination of the generalized Newton method for absolute value equations

A generalization of the Newton-based matrix splitting iteration method for generalized absolute value equations

A Modified Newton-based Matrix Splitting Iteration Method for Generalized Absolute Value Equations

Smale's α-theory for inexact Newton methods under the γ-condition

Newton's Method for Underdetermined Systems of Equations under the Γ-Condition

Convergence Criterion of Newton's Method for Singular Systems with Constant Rank Derivatives

Randomized iterative methods for generalized absolute value equations: Solvability and error bounds

Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

Two effective inexact iteration methods for solving the generalized absolute value equations

On finite termination of quasi-Newton methods on quadratic problems

The Existence the Solution of Nonlinear Discrete Schemes and Convergence of a Linearized Iterative Method for time-dependent PNP Equations

Existence and nonexistence of solutions for underdetermined generalized absolute value equations

Newton's method on two step nilpotent lie groups: convergence criteria and the extended smale point estimate theory

A generalization of the relaxation-based matrix splitting iterative method for solving the system of generalized absolute value equations

A modified fixed point iteration method for solving the system of absolute value equations

ON CONVERGENCE OF A TRUNCATED GAUSS-NEWTON METHOD FOR SOLVING UNDERDETERMINED NONLINEAR LEAST SQUARES PROBLEMS

On Local Behavior of Newton-Type Methods Near Critical Solutions of Constrained Equations

Convergence of block iterative methods for linear systems with generalized H-matrices

Sufficient conditions for the unique solvability of absolute value matrix equations