Abstract:This paper aims to develop a Newton-type method to solve a class of nonconvex composite programs. In particular, the nonsmooth part is possibly nonconvex. To tackle the nonconvexity, we develop a notion of strong prox-regularity which is related to the singleton property, Lipschitz continuity, and monotonicity of the associated proximal operator, and we verify it in various classes of functions, including weakly convex functions, indicator functions of proximally smooth sets, and two specific sphere-related nonconvex nonsmooth functions. In this case, the problem class we are concerned with covers smooth optimization problems on manifold and certain composite optimization problems on manifold. For the latter, the proposed algorithm is the first second-order type method. Combining with the semismoothness of the proximal operator, we design a projected semismooth Newton method to find a root of the natural residual induced by the proximal gradient method. Since the corresponding natural residual may not be globally monotone, an extra projection is added on the usual semismooth Newton step and new criteria are proposed for the switching between the projected semismooth Newton step and the proximal step. The global convergence is then established under the strong prox-regularity. Based on the BD regularity condition, we establish local superlinear convergence. Numerical experiments demonstrate the effectiveness of our proposed method compared with state-of-the-art ones.

A Riemannian Proximal Newton Method

A Riemannian Proximal Newton Method

A Riemannian Proximal Newton-CG Method

An Inexact Riemannian Proximal Gradient Method

A globally convergent proximal Newton-type method in nonsmooth convex optimization

On the linear convergence rate of Riemannian proximal gradient method

The Proximal Gradient Method for Composite Optimization Problems on Riemannian Manifolds

Proximal Quasi-Newton Method for Composite Optimization over the Stiefel Manifold.

Proximal Diagonal Newton Methods for Composite Optimization Problems

Riemannian proximal gradient methods

Proximal methods for structured nonsmooth optimization over Riemannian submanifolds

Convergence and Trade-Offs in Riemannian Gradient Descent and Riemannian Proximal Point

Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants

Riemannian Proximal Gradient Methods (extended Version)

Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold

A projected semismooth Newton method for a class of nonconvex composite programs with strong prox-regularity

A Decentralized Proximal Gradient Tracking Algorithm for Composite Optimization on Riemannian Manifolds

Stochastic Newton Proximal Extragradient Method

Riemannian Linearized Proximal Algorithms for Nonnegative Inverse Eigenvalue Problem.

A Proximal-Gradient Method for Constrained Optimization

An Adaptive Regularized Proximal Newton-Type Methods for Composite Optimization over the Stiefel Manifold