Abstract:We propose a coderivative-based generalized regularized Newton method with positive definite regularization term (GRNM-PD) to solve optimization problems. In GRNM-PD, a general positive definite symmetric matrix is used to regularize the generalized Hessian, in contrast to the recently proposed GRNM, which uses the identity matrix. Our approach features global convergence and fast local convergence rate even for problems with nonisolated solutions. To this end, we introduce the p -order semismooth property which plays the same role in our analysis as Lipschitz continuity of the Hessian does in the case. Imposing only the metric q -subregularity of the gradient at a solution, we establish global convergence of the proposed algorithm as well as its local convergence rate, which can be superlinear, quadratic, or even higher than quadratic, depending on an algorithmic parameter and the regularity parameters p and q . Specifically, choosing to be one, we achieve quadratic local convergence rate under metric subregularity and the strong semismooth property. The algorithm is applied to a class of nonsmooth convex composite minimization problems through the machinery of forward–backward envelope. The greater flexibility in the choice of regularization matrices leads to notable improvement in practical performance. Numerical experiments on box-constrained quadratic programming problems demonstrate the efficiency of our algorithm.

A hybrid inexact regularized Newton and negative curvature method

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

Exploiting Negative Curvature in Conjunction with Adaptive Sampling: Theoretical Results and a Practical Algorithm

Online Learning Guided Curvature Approximation: A Quasi-Newton Method with Global Non-Asymptotic Superlinear Convergence

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Inexact Proximal Cubic Regularized Newton Methods for Convex Optimization

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Yet another fast variant of Newton’s method for nonconvex optimization

Local Convergence of Inexact Methods under the Hölder Condition

Generalized Newton Method with Positive Definite Regularization for Nonsmooth Optimization Problems with Nonisolated Solutions

Gauss–Newton Method for Convex Composite Optimizations on Riemannian Manifolds

Adaptive Quadratically Regularized Newton Method for Riemannian Optimization

Inexact quasi-Newton methods under a relaxed metric regularity assumption

Saving Gradient and Negative Curvature Computations: Finding Local Minima More Efficiently

Inexact Riemannian Gradient Descent Method for Nonconvex Optimization

Super-Universal Regularized Newton Method

Inexact Newton-type Methods for Optimisation with Nonnegativity Constraints

An asymptotical regularization with convex constraints for inverse problems

Adaptive regularized quasi-newton method using inexact first-order information

Minimization Over the Nonconvex Sparsity Constraint Using A Hybrid First-order method