Abstract:The work proves that under conditions of instability of the second derivatives of the function in the minimization region, the estimate of the convergence rate of Newton's method is determined by the parameters of the irreducible part of the conditionality degree of the problem. These parameters represent the degree of difference between eigenvalues of the matrices of the second derivatives in the coordinate system, where this difference is minimal, and the resulting estimate of the convergence rate subsequently acts as a standard. The paper studies the convergence rate of the relaxation subgradient method (RSM) with optimization of the parameters of two-rank correction of metric matrices on smooth strongly convex functions with a Lipschitz gradient without assumptions about the existence of second derivatives of the function. The considered RSM is similar in structure to quasi-Newton minimization methods. Unlike the latter, its metric matrix is not an approximation of the inverse matrix of second derivatives but is adjusted in such a way that it enables one to find the descent direction that takes the method beyond a certain neighborhood of the current minimum as a result of one-dimensional minimization along it. This means that the metric matrix enables one to turn the current gradient into a direction that is gradient-consistent with the set of gradients of some neighborhood of the current minimum. Under broad assumptions on the parameters of transformations of metric matrices, an estimate of the convergence rate of the studied RSM and an estimate of its ability to exclude removable linear background are obtained. The obtained estimates turn out to be qualitatively similar to estimates for Newton's method. In this case, the assumption of the existence of second derivatives of the function is not required. A computational experiment was carried out in which the quasi-Newton BFGS method and the subgradient method under study were compared on various types of smooth functions. The testing results indicate the effectiveness of the subgradient method in minimizing smooth functions with a high degree of conditionality of the problem and its ability to eliminate the linear background that worsens the convergence.

Lipschitz Continuity of the Gradient of a One-Parametric Class of SOC Merit Functions

A Trust Region-Type Method for Solving Monotone Variational Inequality

Convergence Criterion of Newton's Method for Singular Systems with Constant Rank Derivatives

One-parametric class of smoothing functions for (WSOCCP) and its Jacobian consistency

Convergence of Newton'S Method for Systems of Equations with Constant Rank Derivatives

Non-asymptotic convergence bounds for Sinkhorn iterates and their gradients: a coupling approach

Convergence of the momentum method for semialgebraic functions with locally Lipschitz gradients

Revisiting Subgradient Method: Complexity and Convergence Beyond Lipschitz Continuity

Convergence Analysis of a Class of Nonsmooth Gradient Systems.

The gradient's limit of a definable family of functions is a conservative set-valued field

Merit Functions in Vector Optimization

Global convergence of the gradient method for functions definable in o-minimal structures

Smoothness of Subgradient Mappings and Its Applications in Parametric Optimization

Newtonian Property of Subgradient Method with Optimization of Metric Matrix Parameter Correction

Gradient is All You Need?

Newton-CG methods for nonconvex unconstrained optimization with Hölder continuous Hessian

Optimality Conditions for Problems over Symmetric Cones and a Simple Augmented Lagrangian Method

Generalized conditional gradient methods for multiobjective composite optimization problems with H{ö}lder condition

Heaviside Set Constrained Optimization: Optimality and Newton Method

A Continuous-Time Perspective on Global Acceleration for Monotone Equation Problems

A Local Nearly Linearly Convergent First-Order Method for Nonsmooth Functions with Quadratic Growth