Abstract:The convergence analysis of optimization algorithms using continuous-time dynamical systems has received much attention in recent years. In this paper, we investigate applications of these systems to analyze the convergence of linearized proximal ADMM algorithms for nonconvex composite optimization, whose objective function is the sum of a continuously differentiable function and a composition of a possibly nonconvex function with a linear operator. We first derive a first-order differential inclusion for the linearized proximal ADMM algorithm, LP-ADMM. Both the global convergence and the convergence rates of the generated trajectory are established with the use of Kurdyka-Łojasiewicz (KL) property. Then, a stochastic variant, LP-SADMM, is delved into an investigation for finite-sum nonconvex composite problems. Under mild conditions, we obtain the stochastic differential equation corresponding to LP-SADMM, and demonstrate the almost sure global convergence of the generated trajectory by leveraging the KL property. Based on the almost sure convergence of trajectory, we construct a stochastic process that converges almost surely to an approximate critical point of objective function, and derive the expected convergence rates associated with this stochastic process. Moreover, we propose an accelerated LP-SADMM that incorporates Nesterov's acceleration technique. The continuous-time dynamical system of this algorithm is modeled as a second-order stochastic differential equation. Within the context of KL property, we explore the related almost sure convergence and expected convergence rates.

Exponential Convergence of Primal-Dual Dynamical System for Linear Constrained Optimization

Convergence Rate of Primal-Dual Approach to Constrained Reinforcement Learning with Softmax Policy

"Second-Order Primal" + "First-Order Dual" Dynamical Systems With Time Scaling for Linear Equality Constrained Convex Optimization Problems

Tikhonov regularized mixed-order primal-dual dynamical system for convex optimization problems with linear equality constraints

A fast primal-dual algorithm via dynamical system with variable mass for linearly constrained convex optimization

A Second Order Primal–Dual Dynamical System for a Convex–Concave Bilinear Saddle Point Problem

Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems

Tikhonov regularization of second-order plus first-order primal-dual dynamical systems for separable convex optimization

Finite-Time Nonconvex Optimization Using Time-Varying Dynamical Systems

Exponential convergence of distributed optimization for heterogeneous linear multi-agent systems

Accelerated Primal-Dual Projection Neurodynamic Approach With Time Scaling for Linear and Set Constrained Convex Optimization Problems

Primal-Dual Algorithm for Distributed Optimization with Coupled Constraints

Accelerated primal-dual methods with enlarged step sizes and operator learning for nonsmooth optimal control problems

Dynamical convergence analysis for nonconvex linearized proximal ADMM algorithms

Stability of Primal-Dual Gradient Flow Dynamics for Multi-Block Convex Optimization Problems

Stochastic Primal-Dual Algorithms with Faster Convergence Than O(1/√T) for Problems Without Bilinear Structure

Linear Convergence of First- and Zeroth-Order Primal-Dual Algorithms for Distributed Nonconvex Optimization

Linear convergence of a primal-dual algorithm for distributed interval optimization

Distributed Optimization with Projection-free Dynamics

Nearly Optimal Linear Convergence of Stochastic Primal-Dual Methods for Linear Programming