Abstract:In this paper, based a novel primal-dual dynamical model with adaptive scaling parameters and Bregman divergences, we propose new accelerated primal-dual proximal gradient splitting methods for solving bilinear saddle-point problems with provable optimal nonergodic convergence rates. For the first, using the spectral analysis, we show that a naive extension of acceleration model for unconstrained optimization problems to a quadratic game is unstable. Motivated by this, we present an accelerated primal-dual hybrid gradient (APDHG) flow which combines acceleration with careful velocity correction. To work with non-Euclidean distances, we also equip our APDHG model with general Bregman divergences and prove the exponential decay of a Lyapunov function. Then, new primal-dual splitting methods are developed based on proper semi-implicit Euler schemes of the continuous model, and the theoretical convergence rates are nonergodic and optimal with respect to the matrix norms,\, Lipschitz constants and convexity parameters. Thanks to the primal and dual scaling parameters, both the algorithm designing and convergence analysis cover automatically the convex and (partially) strongly convex objectives. Moreover, the use of Bregman divergences not only unifies the standard Euclidean distances and general cases in an elegant way, but also makes our methods more flexible and adaptive to problem-dependent metrics.

A Second Order Primal–Dual Dynamical System for a Convex–Concave Bilinear Saddle Point Problem

Inertial primal-dual dynamics with Hessian-driven damping and Tikhonov regularization for convex-concave bilinear saddle point problems

"Second-Order Primal" + "First-Order Dual" Dynamical Systems With Time Scaling for Linear Equality Constrained Convex Optimization Problems

Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems

Tikhonov regularization of second-order plus first-order primal-dual dynamical systems for separable convex optimization

Accelerated Primal-Dual Proximal Gradient Splitting Methods for Convex-Concave Saddle-Point Problems

A fast primal-dual algorithm via dynamical system with variable mass for linearly constrained convex optimization

Tikhonov regularized mixed-order primal-dual dynamical system for convex optimization problems with linear equality constraints

Fast convergence rates and trajectory convergence of a Tikhonov regularized inertial primal\mbox{-}dual dynamical system with time scaling and vanishing damping

Accelerated Primal-dual Scheme for a Class of Stochastic Nonconvex-concave Saddle Point Problems

Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization

An Inexact Primal-Dual Smoothing Framework for Large-Scale Non-Bilinear Saddle Point Problems

Strong asymptotic convergence of a slowly damped inertial primal-dual dynamical system controlled by a Tikhonov regularization term

Inertial Methods with Viscous and Hessian driven Damping for Non-Convex Optimization

Robust Accelerated Primal-Dual Methods for Computing Saddle Points

Transformed Primal-Dual Methods For Nonlinear Saddle Point Systems

Accelerated Primal-Dual Gradient Method for Smooth and Convex-Concave Saddle-Point Problems with Bilinear Coupling

Primal-dual Accelerated Mirror-Descent Method for Constrained Bilinear Saddle-Point Problems