Abstract:We consider a class of nonconvex nonsmooth optimization problems whose objective is the sum of a smooth function and a finite number of nonnegative proper closed possibly nonsmooth functions (whose proximal mappings are easy to compute), some of which are further composed with linear maps. This kind of problems arises naturally in various applications when different regularizers are introduced for inducing simultaneous structures in the solutions. Solving these problems, however, can be challenging because of the coupled nonsmooth functions: the corresponding proximal mapping can be hard to compute so that standard first-order methods such as the proximal gradient algorithm cannot be applied efficiently. In this paper, we propose a successive difference-of-convex approximation method for solving this kind of problems. In this algorithm, we approximate the nonsmooth functions by their Moreau envelopes in each iteration. Making use of the simple observation that Moreau envelopes of nonnegative proper closed functions are continuous difference-of-convex functions, we can then approximately minimize the approximation function by first-order methods with suitable majorization techniques. These first-order methods can be implemented efficiently thanks to the fact that the proximal mapping of each nonsmooth function is easy to compute. Under suitable assumptions, we prove that the sequence generated by our method is bounded and any accumulation point is a stationary point of the objective. We also discuss how our method can be applied to concrete applications such as nonconvex fused regularized optimization problems and simultaneously structured matrix optimization problems, and illustrate the performance numerically for these two specific applications.

An Approximated Proximal Alternating Method for Non-convex and Non-smooth Optimization

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Efficient Inexact Proximal Gradient Algorithm for Nonconvex Problems.

A successive difference-of-convex approximation method for a class of nonconvex nonsmooth optimization problems

Semi-Proximal Point Method for Nonsmooth Convex-Concave Minimax Optimization

A Proximal Alternating Linearization Method for Nonconvex Optimization Problems

More Efficient Accelerated Proximal Algorithm for Nonconvex Problems

Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

An approximation proximal gradient algorithm for nonconvex-linear minimax problems with nonconvex nonsmooth terms

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Adaptive Proximal Average Approximation for Composite Convex Minimization.

Approximate Bregman Proximal Gradient Algorithm for Relatively Smooth Nonconvex Optimization

An Accelerated Distributed Method with Inexact Model of Relative Smoothness and Strong Convexity

Accelerated Proximal Gradient Methods for Nonconvex Programming

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

An Inexact Preconditioned Zeroth-order Proximal Method for Composite Optimization

A Note on the Accelerated Proximal Gradient Method for Nonconvex Optimization

An Adaptive Proximal Bundle Method with Inexact Oracles for a Class of Nonconvex and Nonsmooth Composite Optimization

An Optimization Framework with Flexible Inexact Inner Iterations for Nonconvex and Nonsmooth Programming

A Symplectic Discretization Based Proximal Point Algorithm for Convex Minimization