Abstract:We consider the problem of minimizing the sum of a smooth function $h$ with a bounded Hessian and a nonsmooth function. We assume that the latter function is a composition of a proper closed function $P$ and a surjective linear map $\mathcal{M}$, with the proximal mappings of $\tau P$, $\tau > 0$, simple to compute. This problem is nonconvex in general and encompasses many important applications in engineering and machine learning. In this paper, we examined two types of splitting methods for solving this nonconvex optimization problem: the alternating direction method of multipliers and the proximal gradient algorithm. For the direct adaptation of the alternating direction method of multipliers, we show that if the penalty parameter is chosen sufficiently large and the sequence generated has a cluster point, then it gives a stationary point of the nonconvex problem. We also establish convergence of the whole sequence under an additional assumption that the functions $h$ and $P$ are semialgebraic. Furthermore, we give simple sufficient conditions to guarantee boundedness of the sequence generated. These conditions can be satisfied for a wide range of applications including the least squares problem with the $\ell_{1/2}$ regularization. Finally, when $\mathcal{M}$ is the identity so that the proximal gradient algorithm can be efficiently applied, we show that any cluster point is stationary under a slightly more flexible constant step-size rule than what is known in the literature for a nonconvex $h$.

On the Convergence Rates of Proximal Splitting Algorithms

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

Convergence Rates of Forward–Douglas–Rachford Splitting Method

Convergence Rates of Inertial Splitting Schemes for Nonconvex Composite Optimization

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Local Linear Convergence Analysis of Primal–Dual Splitting Methods

Convergence of the Preconditioned Proximal Point Method and Douglas-Rachford Splitting in the Absence of Monotonicity

Linear Convergence Rate of Splitting Algorithms for Multi-Block Constrained Convex Minimizations

Improved Convergence Bounds For Operator Splitting Algorithms With Rare Extreme Errors

Global Convergence of Splitting Methods for Nonconvex Composite Optimization

Local Convergence Properties of Douglas–Rachford and Alternating Direction Method of Multipliers

Finite Termination of Inexact Proximal Point Algorithms in Hilbert Spaces

Iterative algorithms for equilibrium problems based on proximal-like methods

A Forward-Backward Bregman Splitting Scheme for Regularized Distributed Optimization Problems

The splitting algorithms by Ryu, by Malitsky-Tam, and by Campoy applied to normal cones of linear subspaces converge strongly to the projection onto the intersection

Algorithm 1 : Non-stationary Forward – Douglas – Rachford splitting

The Iterative Method for Solving the Proximal Split Feasibility Problem with an Application to LASSO Problem.

Convergence of Generalized Proximal Point Algorithms

Strong convergence of Bregman projection algorithms for solving split feasibility problems