Abstract:We consider the problem of minimizing the sum of a smooth function $h$ with a bounded Hessian and a nonsmooth function. We assume that the latter function is a composition of a proper closed function $P$ and a surjective linear map $\mathcal{M}$, with the proximal mappings of $\tau P$, $\tau > 0$, simple to compute. This problem is nonconvex in general and encompasses many important applications in engineering and machine learning. In this paper, we examined two types of splitting methods for solving this nonconvex optimization problem: the alternating direction method of multipliers and the proximal gradient algorithm. For the direct adaptation of the alternating direction method of multipliers, we show that if the penalty parameter is chosen sufficiently large and the sequence generated has a cluster point, then it gives a stationary point of the nonconvex problem. We also establish convergence of the whole sequence under an additional assumption that the functions $h$ and $P$ are semialgebraic. Furthermore, we give simple sufficient conditions to guarantee boundedness of the sequence generated. These conditions can be satisfied for a wide range of applications including the least squares problem with the $\ell_{1/2}$ regularization. Finally, when $\mathcal{M}$ is the identity so that the proximal gradient algorithm can be efficiently applied, we show that any cluster point is stationary under a slightly more flexible constant step-size rule than what is known in the literature for a nonconvex $h$.

On the convergence of partitioning group correction algorithms

Partitioning group correction Cholesky techniques for large scale sparse unconstrained optimization

A GLOBALLY AND SUPERLIEARLY CONVERGENT FEASIBLE SQP ALGORITHM FOR DEGENERATE CONSTRAINED OPTIMIZATION

The Substitution Secant/Finite Difference Method for Large Scale Sparse Unconstrained Optimization

Convergence Rate of the Relaxed CQ Algorithm under Hölderian Type Error Bound Property

Convergence depth control for interior point methods

Convergence of a class of algorithms with the level-set subdifferential error bound

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

The Convergence of Krylov Subspace Methods for Large Unsymmetric Linear Systems

Convergence of accelerated modulus-based matrix splitting iteration methods for linear complementarity problem with an H + -matrix

Global Convergence of Splitting Methods for Nonconvex Composite Optimization

A Parallel Line Search Subspace Correction Method for Composite Convex Optimization

On the Convergence of Two-Step Modulus-Based Matrix Splitting Iteration Methods for a Restricted Class of Nonlinear Complementarity Problems with H+-Matrices

A Nonconvex Implementation of Sparse Subspace Clustering: Algorithm and Convergence Analysis

On the Convergence of Modulus-Based Matrix Splitting Iteration Methods for a Class of Nonlinear Complementarity Problems with H+-matrices

Convergent analysis for block iterative methods for H-matrix linear complementarity problems

Convergence of Accelerated Modulus-Based Matrix Splitting Iteration Methods for Linear Complementarity Problem with an H+-Matrix

A superlinearly convergent SQP algorithm for constrained optimization problems

On the Convergence of Subproper (Multi)-Splitting Methods for Solving Rectangular Linear Systems

Linear Convergence Rate of Splitting Algorithms for Multi-Block Constrained Convex Minimizations

Convergence of the Augmented Decomposition Algorithm