Abstract:The work proves that under conditions of instability of the second derivatives of the function in the minimization region, the estimate of the convergence rate of Newton's method is determined by the parameters of the irreducible part of the conditionality degree of the problem. These parameters represent the degree of difference between eigenvalues of the matrices of the second derivatives in the coordinate system, where this difference is minimal, and the resulting estimate of the convergence rate subsequently acts as a standard. The paper studies the convergence rate of the relaxation subgradient method (RSM) with optimization of the parameters of two-rank correction of metric matrices on smooth strongly convex functions with a Lipschitz gradient without assumptions about the existence of second derivatives of the function. The considered RSM is similar in structure to quasi-Newton minimization methods. Unlike the latter, its metric matrix is not an approximation of the inverse matrix of second derivatives but is adjusted in such a way that it enables one to find the descent direction that takes the method beyond a certain neighborhood of the current minimum as a result of one-dimensional minimization along it. This means that the metric matrix enables one to turn the current gradient into a direction that is gradient-consistent with the set of gradients of some neighborhood of the current minimum. Under broad assumptions on the parameters of transformations of metric matrices, an estimate of the convergence rate of the studied RSM and an estimate of its ability to exclude removable linear background are obtained. The obtained estimates turn out to be qualitatively similar to estimates for Newton's method. In this case, the assumption of the existence of second derivatives of the function is not required. A computational experiment was carried out in which the quasi-Newton BFGS method and the subgradient method under study were compared on various types of smooth functions. The testing results indicate the effectiveness of the subgradient method in minimizing smooth functions with a high degree of conditionality of the problem and its ability to eliminate the linear background that worsens the convergence.

On the Redundancy of Hessian Nonsingularity for Linear Convergence Rate of the Newton Method Applied to the Minimization of Convex Functions

Convergence Criterion of Newton's Method for Singular Systems with Constant Rank Derivatives

Convergence of Newton'S Method for Systems of Equations with Constant Rank Derivatives

Gauss-Newton Methods for A Class of Nonsmooth Optimization Problems

A Regularized Newton Method for Monotone Nonlinear Equations and Its Application

Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

Newton and interior-point methods for (constrained) nonconvex-nonconcave minmax optimization with stability and instability guarantees

On Convergence of the Gauss-Newton Method for Convex Composite Optimization

Newton-type Methods for Inverse Singular Value Problems with Multiple Singular Values

Majorizing Functions and Convergence of the Gauss-Newton Method for Convex Composite Optimization

Fast convergence to non-isolated minima: four equivalent conditions for $\mathrm{C}^2$ functions

Convergence of Newton'S Method and Uniqueness of the Solution of Equations in Banach Spaces Ii

Convergence and Uniqueness Properties of Gauss-Newton's Method

Local Convergence of Inexact Methods under the Hölder Condition

Monotone vector fields and the proximal point algorithm on Hadamard manifolds

On Local Behavior of Newton-Type Methods Near Critical Solutions of Constrained Equations

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

A new convergence rate of the steepest descent regarding the Euclidean norm

Yet another fast variant of Newton’s method for nonconvex optimization

Local minima in Newton's aerodynamical problem and inequalities between norms of partial derivatives

Newtonian Property of Subgradient Method with Optimization of Metric Matrix Parameter Correction