Abstract:The convergence analysis of optimization algorithms using continuous-time dynamical systems has received much attention in recent years. In this paper, we investigate applications of these systems to analyze the convergence of linearized proximal ADMM algorithms for nonconvex composite optimization, whose objective function is the sum of a continuously differentiable function and a composition of a possibly nonconvex function with a linear operator. We first derive a first-order differential inclusion for the linearized proximal ADMM algorithm, LP-ADMM. Both the global convergence and the convergence rates of the generated trajectory are established with the use of Kurdyka-Łojasiewicz (KL) property. Then, a stochastic variant, LP-SADMM, is delved into an investigation for finite-sum nonconvex composite problems. Under mild conditions, we obtain the stochastic differential equation corresponding to LP-SADMM, and demonstrate the almost sure global convergence of the generated trajectory by leveraging the KL property. Based on the almost sure convergence of trajectory, we construct a stochastic process that converges almost surely to an approximate critical point of objective function, and derive the expected convergence rates associated with this stochastic process. Moreover, we propose an accelerated LP-SADMM that incorporates Nesterov's acceleration technique. The continuous-time dynamical system of this algorithm is modeled as a second-order stochastic differential equation. Within the context of KL property, we explore the related almost sure convergence and expected convergence rates.

A Bregman-Style Improved ADMM and its Linearized Version in the Nonconvex Setting: Convergence and Rate Analyses

Linearized ADMM for Nonconvex Nonsmooth Optimization with Convergence Analysis

A Bregman-style Partially Symmetric Alternating Direction Method of Multipliers for Nonconvex Multi-block Optimization

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

An Inertial Proximal Partially Symmetric ADMM-based Algorithm for Linearly Constrained Multi-Block Nonconvex Optimization Problems with Applications.

Linearized Symmetric Multi-Block ADMM with Indefinite Proximal Regularization and Optimal Proximal Parameter

CONVERGENCE OF ADMM FOR OPTIMIZATION PROBLEMS WITH NONSEPARABLE NONCONVEX OBJECTIVE AND LINEAR CONSTRAINTS

Linearized Block-Wise Alternating Direction Method of Multipliers for Multiple-Block Convex Programming

The Convergence Rate Analysis of the Symmetric ADMM for the Nonconvex Separable Optimization Problems

A Note on the Convergence of ADMM for Linearly Constrained Convex Optimization Problems

A general analysis of the convergence of ADMM

Linear Rate Convergence of the Alternating Direction Method of Multipliers for Convex Composite Programming.

Sublinear and Linear Convergence of Modified ADMM for Distributed Nonconvex Optimization

Dynamical convergence analysis for nonconvex linearized proximal ADMM algorithms

Sequential inertial linear ADMM algorithm for nonconvex and nonsmooth multiblock problems with nonseparable structure

A partial Bregman ADMM with a general relaxation factor for structured nonconvex and nonsmooth optimization

Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems

Linearized Alternating Direction Method of Multipliers for Constrained Nonconvex Regularized Optimization.

Local Linear Convergence of an ADMM-Type Splitting Framework for Equality Constrained Optimization

Stochastic linearized generalized alternating direction method of multipliers: Expected convergence rates and large deviation properties

Convergence of Multi-Block Bregman ADMM for Nonconvex Composite Problems