Abstract:We consider a class of nonconvex nonsmooth optimization problems whose objective is the sum of a smooth function and a finite number of nonnegative proper closed possibly nonsmooth functions (whose proximal mappings are easy to compute), some of which are further composed with linear maps. This kind of problems arises naturally in various applications when different regularizers are introduced for inducing simultaneous structures in the solutions. Solving these problems, however, can be challenging because of the coupled nonsmooth functions: the corresponding proximal mapping can be hard to compute so that standard first-order methods such as the proximal gradient algorithm cannot be applied efficiently. In this paper, we propose a successive difference-of-convex approximation method for solving this kind of problems. In this algorithm, we approximate the nonsmooth functions by their Moreau envelopes in each iteration. Making use of the simple observation that Moreau envelopes of nonnegative proper closed functions are continuous difference-of-convex functions, we can then approximately minimize the approximation function by first-order methods with suitable majorization techniques. These first-order methods can be implemented efficiently thanks to the fact that the proximal mapping of each nonsmooth function is easy to compute. Under suitable assumptions, we prove that the sequence generated by our method is bounded and any accumulation point is a stationary point of the objective. We also discuss how our method can be applied to concrete applications such as nonconvex fused regularized optimization problems and simultaneously structured matrix optimization problems, and illustrate the performance numerically for these two specific applications.

A linearly convergent method for solving high-order proximal operator

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

A globally convergent proximal Newton-type method in nonsmooth convex optimization

A Proximal Alternating Linearization Method for Nonconvex Optimization Problems

Iterative algorithms for equilibrium problems based on proximal-like methods

A Proximal-Gradient Method for Constrained Optimization

Linear Convergence of the Proximal Gradient Method for Composite Optimization Under the Polyak-Łojasiewicz Inequality and Its Variant

A Proximal Iteratively Regularized Gauss–Newton Method for Nonlinear Inverse Problems

Proximal Dogleg Opportunistic Majorization for Nonconvex and Nonsmooth Optimization

Strong convergence for the proximal-gradient method

Linear Convergence of the Derivative-Free Proximal Bundle Method on Convex Nonsmooth Functions, with Application to the Derivative-Free -Algorithm

A Class of Nonlinear Proximal Point Algorithms for Variational Inequality Problems

The Rate of Convergence of Proximal Method of Multipliers for Nonlinear Programming.

A successive difference-of-convex approximation method for a class of nonconvex nonsmooth optimization problems

A projected semismooth Newton method for a class of nonconvex composite programs with strong prox-regularity

On the Convergence of Newton-type Proximal Gradient Method for Multiobjective Optimization Problems

An adaptive proximal point algorithm framework and application to large-scale optimization

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs

The Rate of Convergence of Proximal Method of Multipliers for Equality Constrained Optimization Problems.

Directional proximal point method for convex optimization