Abstract:We consider a class of nonconvex nonsmooth optimization problems whose objective is the sum of a smooth function and a finite number of nonnegative proper closed possibly nonsmooth functions (whose proximal mappings are easy to compute), some of which are further composed with linear maps. This kind of problems arises naturally in various applications when different regularizers are introduced for inducing simultaneous structures in the solutions. Solving these problems, however, can be challenging because of the coupled nonsmooth functions: the corresponding proximal mapping can be hard to compute so that standard first-order methods such as the proximal gradient algorithm cannot be applied efficiently. In this paper, we propose a successive difference-of-convex approximation method for solving this kind of problems. In this algorithm, we approximate the nonsmooth functions by their Moreau envelopes in each iteration. Making use of the simple observation that Moreau envelopes of nonnegative proper closed functions are continuous difference-of-convex functions, we can then approximately minimize the approximation function by first-order methods with suitable majorization techniques. These first-order methods can be implemented efficiently thanks to the fact that the proximal mapping of each nonsmooth function is easy to compute. Under suitable assumptions, we prove that the sequence generated by our method is bounded and any accumulation point is a stationary point of the objective. We also discuss how our method can be applied to concrete applications such as nonconvex fused regularized optimization problems and simultaneously structured matrix optimization problems, and illustrate the performance numerically for these two specific applications.

A Hybrid Bregman Alternating Direction Method of Multipliers for the Linearly Constrained Difference-of-convex Problems

A Homotopy Alternating Direction Method of Multipliers for Linearly Constrained Separable Convex Optimization

Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints

On the Convergence of a Generalized Alternating Direction Method of Multipliers with Semi-Proximal Terms

Local Linear Convergence of the Alternating Direction Method of Multipliers for Nonconvex Separable Optimization Problems

The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates

A Proximal Alternating Direction Method of Multipliers for a Minimization Problem with Nonconvex Constraints

An Inertial Proximal Alternating Direction Method of Multipliers for Nonconvex Optimization.

Alternating Direction Method of Multipliers with Difference of Convex Functions.

Alternating Direction Method of Multipliers for a Class of Nonconvex Bilinear Optimization: Convergence Analysis and Applications.

A Generalized Alternating Direction Method of Multipliers with Semi-Proximal Terms for Convex Composite Conic Programming

A successive difference-of-convex approximation method for a class of nonconvex nonsmooth optimization problems

A Bregman proximal subgradient algorithm for nonconvex and nonsmooth fractional optimization problems

Convergence Analysis on a Modified Generalized Alternating Direction Method of Multipliers

Iteratively Linearized Reweighted Alternating Direction Method of Multipliers for a Class of Nonconvex Problems

A Stochastic Alternating Direction Method of Multipliers for Non-smooth and Non-convex Optimization

Alternating Direction Method of Multipliers with Variable Metric Indefinite Proximal Terms for Convex Optimization

A Bregman-style Partially Symmetric Alternating Direction Method of Multipliers for Nonconvex Multi-block Optimization

A Symmetric Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problems

An Alternating Direction Method of Multipliers with the BFGS Update for Structured Convex Quadratic Optimization

Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems