Abstract:In this paper, we study the local linear convergence properties of a versatile class of Primal-Dual splitting methods for minimizing composite nonsmooth convex optimization problems. Under the assumption that the nonsmooth components of the problem are partly smooth relative to smooth manifolds, we present a unified local convergence analysis framework for these methods. More precisely, in our framework, we first show that (i) the sequences generated by Primal-Dual splitting methods identify a pair of primal and dual smoothmanifolds in a finite number of iterations, and then (ii) enter a local linear convergence regime, which is characterized based on the structure of the underlying active smoothmanifolds. We also show how our results for Primal-Dual splitting canbe specialized to cover existing ones on Forward-Backward splitting and Douglas-Rachford splitting/ADMM (alternating direction methods of multipliers). Moreover, based on these obtained local convergence analysis result, several practical acceleration techniques are discussed. To exemplify the usefulness of the obtained result, we consider several concrete numerical experiments arising from fields including signal/image processing, inverse problems and machine learning. The demonstration not only verifies the local linear convergence behaviour of Primal-Dual splitting methods, but also the insights on how to accelerate them in practice.

Convergence of the Preconditioned Proximal Point Method and Douglas-Rachford Splitting in the Absence of Monotonicity

Weak and Strong Convergence of Split Douglas-Rachford Algorithms for Monotone Inclusions

On the Bredies-Chenchene-Lorenz-Naldi algorithm

Local Convergence Properties of Douglas–Rachford and Alternating Direction Method of Multipliers

A Proximal Point Analysis of the Preconditioned Alternating Direction Method of Multipliers.

Anisotropic Proximal Point Algorithm

On the Convergence Rates of Proximal Splitting Algorithms

Convergence of the Chambolle-Pock Algorithm in the Absence of Monotonicity

Convergence of the Peaceman-Rachford Splitting Method for a Class of Nonconvex Programs

A splitting primal-dual proximity algorithm for solving composite optimization problems

Convergence Analysis of Douglas--Rachford Splitting Method for “strongly + Weakly” Convex Programming

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Local Linear Convergence Analysis of Primal–Dual Splitting Methods

Understanding the Convergence of the Preconditioned PDHG Method: A View of Indefinite Proximal ADMM

Convergence Rates of Forward–Douglas–Rachford Splitting Method

The splitting algorithms by Ryu, by Malitsky-Tam, and by Campoy applied to normal cones of linear subspaces converge strongly to the projection onto the intersection

A stochastic preconditioned Douglas-Rachford splitting method for saddle-point problems

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

On the convergence of the distributed proximal point algorithm

Preconditioned Douglas-Rachford Splitting Methods for Convex-concave Saddle-point Problems.

Linear Convergence of Resolvent Splitting with Minimal Lifting and its Application to a Primal-Dual Algorithm