Abstract:The convergence analysis of optimization algorithms using continuous-time dynamical systems has received much attention in recent years. In this paper, we investigate applications of these systems to analyze the convergence of linearized proximal ADMM algorithms for nonconvex composite optimization, whose objective function is the sum of a continuously differentiable function and a composition of a possibly nonconvex function with a linear operator. We first derive a first-order differential inclusion for the linearized proximal ADMM algorithm, LP-ADMM. Both the global convergence and the convergence rates of the generated trajectory are established with the use of Kurdyka-Łojasiewicz (KL) property. Then, a stochastic variant, LP-SADMM, is delved into an investigation for finite-sum nonconvex composite problems. Under mild conditions, we obtain the stochastic differential equation corresponding to LP-SADMM, and demonstrate the almost sure global convergence of the generated trajectory by leveraging the KL property. Based on the almost sure convergence of trajectory, we construct a stochastic process that converges almost surely to an approximate critical point of objective function, and derive the expected convergence rates associated with this stochastic process. Moreover, we propose an accelerated LP-SADMM that incorporates Nesterov's acceleration technique. The continuous-time dynamical system of this algorithm is modeled as a second-order stochastic differential equation. Within the context of KL property, we explore the related almost sure convergence and expected convergence rates.

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Linear Convergence of the Proximal Gradient Method for Composite Optimization Under the Polyak-Łojasiewicz Inequality and Its Variant

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

A linearly convergent method for solving high-order proximal operator

A globally convergent proximal Newton-type method in nonsmooth convex optimization

Variational Analysis Perspective on Linear Convergence of Some First Order Methods for Nonsmooth Convex Optimization Problems

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

On Convergence of the Gauss-Newton Method for Convex Composite Optimization

Convergence of Nonmonotone Proximal Gradient Methods under the Kurdyka-Lojasiewicz Property without a Global Lipschitz Assumption

An adaptive proximal point algorithm framework and application to large-scale optimization

An Inexact Preconditioned Zeroth-order Proximal Method for Composite Optimization

Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

On the linear convergence rate of Riemannian proximal gradient method

An inexact LPA for DC composite optimization and application to matrix completions with outliers

Dynamical convergence analysis for nonconvex linearized proximal ADMM algorithms

The Convergence Properties of Infeasible Inexact Proximal Alternating Linearized Minimization

Global non-asymptotic super-linear convergence rates of regularized proximal quasi-Newton methods on non-smooth composite problems

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs