Abstract:This paper aims to develop a Newton-type method to solve a class of nonconvex composite programs. In particular, the nonsmooth part is possibly nonconvex. To tackle the nonconvexity, we develop a notion of strong prox-regularity which is related to the singleton property, Lipschitz continuity, and monotonicity of the associated proximal operator, and we verify it in various classes of functions, including weakly convex functions, indicator functions of proximally smooth sets, and two specific sphere-related nonconvex nonsmooth functions. In this case, the problem class we are concerned with covers smooth optimization problems on manifold and certain composite optimization problems on manifold. For the latter, the proposed algorithm is the first second-order type method. Combining with the semismoothness of the proximal operator, we design a projected semismooth Newton method to find a root of the natural residual induced by the proximal gradient method. Since the corresponding natural residual may not be globally monotone, an extra projection is added on the usual semismooth Newton step and new criteria are proposed for the switching between the projected semismooth Newton step and the proximal step. The global convergence is then established under the strong prox-regularity. Based on the BD regularity condition, we establish local superlinear convergence. Numerical experiments demonstrate the effectiveness of our proposed method compared with state-of-the-art ones.

Accelerated Proximal Subsampled Newton Method

An Adaptive Regularized Proximal Newton-Type Methods for Composite Optimization over the Stiefel Manifold

A Proximal Stochastic Quasi-Newton Algorithm with Dynamical Sampling and Stochastic Line Search

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization

Accelerated Proximal Coordinate Gradient Methods

Proximal Diagonal Newton Methods for Composite Optimization Problems

A projected semismooth Newton method for a class of nonconvex composite programs with strong prox-regularity

An Inexact Preconditioned Zeroth-order Proximal Method for Composite Optimization

A Riemannian Proximal Newton Method

Nesterov's Acceleration For Approximate Newton.

Revisiting Sub-sampled Newton Methods

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

N ov 2 01 6 A Proximal Stochastic Quasi-Newton Algorithm

More Efficient Accelerated Proximal Algorithm for Nonconvex Problems

Adaptive Proximal Average Approximation for Composite Convex Minimization.

Proximal Gradient/Semismooth Newton Methods for Projection Onto a Polyhedron Via the Duality-Gap-Active-Set Strategy

A Symplectic Discretization Based Proximal Point Algorithm for Convex Minimization

Accelerated Proximal Gradient Methods for Nonconvex Programming

Distributed Accelerated Proximal Coordinate Gradient Methods