Abstract:We consider a class of nonconvex nonsmooth optimization problems whose objective is the sum of a smooth function and a finite number of nonnegative proper closed possibly nonsmooth functions (whose proximal mappings are easy to compute), some of which are further composed with linear maps. This kind of problems arises naturally in various applications when different regularizers are introduced for inducing simultaneous structures in the solutions. Solving these problems, however, can be challenging because of the coupled nonsmooth functions: the corresponding proximal mapping can be hard to compute so that standard first-order methods such as the proximal gradient algorithm cannot be applied efficiently. In this paper, we propose a successive difference-of-convex approximation method for solving this kind of problems. In this algorithm, we approximate the nonsmooth functions by their Moreau envelopes in each iteration. Making use of the simple observation that Moreau envelopes of nonnegative proper closed functions are continuous difference-of-convex functions, we can then approximately minimize the approximation function by first-order methods with suitable majorization techniques. These first-order methods can be implemented efficiently thanks to the fact that the proximal mapping of each nonsmooth function is easy to compute. Under suitable assumptions, we prove that the sequence generated by our method is bounded and any accumulation point is a stationary point of the objective. We also discuss how our method can be applied to concrete applications such as nonconvex fused regularized optimization problems and simultaneously structured matrix optimization problems, and illustrate the performance numerically for these two specific applications.

An extrapolated fixed-point optimization method for strongly convex smooth optimizations

Extrapolated Sequential Constraint Method for Variational Inequality over the Intersection of Fixed-Point Sets

A Proximal-Gradient Method for Constrained Optimization

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems

O(log T) Projections for Stochastic Optimization of Smooth and Strongly Convex Functions

Proximal Point Algorithms for Nonsmooth Convex Optimization with Fixed Point Constraints

A feasible smoothing accelerated projected gradient method for nonsmooth convex optimization

A successive difference-of-convex approximation method for a class of nonconvex nonsmooth optimization problems

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

Fixed point quasiconvex subgradient method

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs

Proximal Dogleg Opportunistic Majorization for Nonconvex and Nonsmooth Optimization

An Infeasible-Point Subgradient Method Using Adaptive Approximate Projections

Exponentially Convergent Algorithm Design for Constrained Distributed Optimization via Non-smooth Approach

Two Stochastic Optimization Algorithms for Convex Optimization with Fixed Point Constraints

Line Search Fixed Point Algorithms Based on Nonlinear Conjugate Gradient Directions: Application to Constrained Smooth Convex Optimization

A modified inertial proximal minimization algorithm for structured nonconvex and nonsmooth problem

A Single-Loop Stochastic Proximal Quasi-Newton Method for Large-Scale Nonsmooth Convex Optimization

A Penalty-Free Infeasible Approach for a Class of Nonsmooth Optimization Problems Over the Stiefel Manifold

Two efficient gradient methods with approximately optimal stepsizes based on regularization models for unconstrained optimization

N ov 2 01 6 A Proximal Stochastic Quasi-Newton Algorithm