Abstract:In this paper, based a novel primal-dual dynamical model with adaptive scaling parameters and Bregman divergences, we propose new accelerated primal-dual proximal gradient splitting methods for solving bilinear saddle-point problems with provable optimal nonergodic convergence rates. For the first, using the spectral analysis, we show that a naive extension of acceleration model for unconstrained optimization problems to a quadratic game is unstable. Motivated by this, we present an accelerated primal-dual hybrid gradient (APDHG) flow which combines acceleration with careful velocity correction. To work with non-Euclidean distances, we also equip our APDHG model with general Bregman divergences and prove the exponential decay of a Lyapunov function. Then, new primal-dual splitting methods are developed based on proper semi-implicit Euler schemes of the continuous model, and the theoretical convergence rates are nonergodic and optimal with respect to the matrix norms,\, Lipschitz constants and convexity parameters. Thanks to the primal and dual scaling parameters, both the algorithm designing and convergence analysis cover automatically the convex and (partially) strongly convex objectives. Moreover, the use of Bregman divergences not only unifies the standard Euclidean distances and general cases in an elegant way, but also makes our methods more flexible and adaptive to problem-dependent metrics.

Exponential Convergence of Primal-Dual Dynamics under General Conditions and Its Application to Distributed Optimization

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

Exponential Convergence of Augmented Primal-dual Gradient Algorithms for Partially Strongly Convex Functions

Linear Convergence of First- and Zeroth-Order Primal-Dual Algorithms for Distributed Nonconvex Optimization

Primal-Dual Algorithm for Distributed Optimization with Coupled Constraints

Linear Convergence of Primal-Dual Gradient Methods and their Performance in Distributed Optimization

Linear Convergence of Distributed Aggregative Optimization with Coupled Inequality Constraints

Tikhonov regularized mixed-order primal-dual dynamical system for convex optimization problems with linear equality constraints

Linear convergence of a primal-dual algorithm for distributed interval optimization

Distributed Optimization with Projection-free Dynamics

Distributed Primal-Dual Proximal Algorithms for Convex Optimization Involving Three Composite Functions

Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems

Computational Convergence Analysis of Distributed Gradient Tracking for Smooth Convex Optimization Using Dissipativity Theory

Stability of Primal-Dual Gradient Flow Dynamics for Multi-Block Convex Optimization Problems

Accelerated Primal-Dual Proximal Gradient Splitting Methods for Convex-Concave Saddle-Point Problems

Stochastic Primal-Dual Algorithms with Faster Convergence Than O(1/√T) for Problems Without Bilinear Structure

Convergence in High Probability of Distributed Stochastic Gradient Descent Algorithms

"Second-Order Primal" + "First-Order Dual" Dynamical Systems With Time Scaling for Linear Equality Constrained Convex Optimization Problems

Distributed Event-Triggered Algorithm for Convex Optimization with Coupled Constraints

A Unified Distributed Method for Constrained Networked Optimization via Saddle-Point Dynamics