Abstract:The convergence analysis of optimization algorithms using continuous-time dynamical systems has received much attention in recent years. In this paper, we investigate applications of these systems to analyze the convergence of linearized proximal ADMM algorithms for nonconvex composite optimization, whose objective function is the sum of a continuously differentiable function and a composition of a possibly nonconvex function with a linear operator. We first derive a first-order differential inclusion for the linearized proximal ADMM algorithm, LP-ADMM. Both the global convergence and the convergence rates of the generated trajectory are established with the use of Kurdyka-Łojasiewicz (KL) property. Then, a stochastic variant, LP-SADMM, is delved into an investigation for finite-sum nonconvex composite problems. Under mild conditions, we obtain the stochastic differential equation corresponding to LP-SADMM, and demonstrate the almost sure global convergence of the generated trajectory by leveraging the KL property. Based on the almost sure convergence of trajectory, we construct a stochastic process that converges almost surely to an approximate critical point of objective function, and derive the expected convergence rates associated with this stochastic process. Moreover, we propose an accelerated LP-SADMM that incorporates Nesterov's acceleration technique. The continuous-time dynamical system of this algorithm is modeled as a second-order stochastic differential equation. Within the context of KL property, we explore the related almost sure convergence and expected convergence rates.

Activity Identification and Local Linear Convergence of Douglas–Rachford/ADMM under Partial Smoothness

Dynamical convergence analysis for nonconvex linearized proximal ADMM algorithms

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Local Linear Convergence of an ADMM-Type Splitting Framework for Equality Constrained Optimization

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

A Bregman-Style Improved ADMM and its Linearized Version in the Nonconvex Setting: Convergence and Rate Analyses

Linearized ADMM for Nonconvex Nonsmooth Optimization with Convergence Analysis

Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

ADMM for Nonconvex Optimization under Minimal Continuity Assumption

Sublinear and Linear Convergence of Modified ADMM for Distributed Nonconvex Optimization

Bregman Proximal Linearized ADMM for Minimizing Separable Sums Coupled by a Difference of Functions

Variational Analysis Perspective on Linear Convergence of Some First Order Methods for Nonsmooth Convex Optimization Problems

Convergence Analysis of Generalized ADMM with Majorization for Linearly Constrained Composite Convex Optimization

On Partly Smoothness, Activity Identification and Faster Algorithms of L 1 over L 2 Minimization

A Convergence Analysis of Nonlinearly Constrained ADMM in Deep Learning

On Partial Smoothness, Activity Identification and Faster Algorithms of L1 over L2 Minimization