Abstract:In this paper, based a novel primal-dual dynamical model with adaptive scaling parameters and Bregman divergences, we propose new accelerated primal-dual proximal gradient splitting methods for solving bilinear saddle-point problems with provable optimal nonergodic convergence rates. For the first, using the spectral analysis, we show that a naive extension of acceleration model for unconstrained optimization problems to a quadratic game is unstable. Motivated by this, we present an accelerated primal-dual hybrid gradient (APDHG) flow which combines acceleration with careful velocity correction. To work with non-Euclidean distances, we also equip our APDHG model with general Bregman divergences and prove the exponential decay of a Lyapunov function. Then, new primal-dual splitting methods are developed based on proper semi-implicit Euler schemes of the continuous model, and the theoretical convergence rates are nonergodic and optimal with respect to the matrix norms,\, Lipschitz constants and convexity parameters. Thanks to the primal and dual scaling parameters, both the algorithm designing and convergence analysis cover automatically the convex and (partially) strongly convex objectives. Moreover, the use of Bregman divergences not only unifies the standard Euclidean distances and general cases in an elegant way, but also makes our methods more flexible and adaptive to problem-dependent metrics.

A subgradient splitting algorithm for optimization on nonpositively curved metric spaces

Horoballs and the subgradient method

Basic Convex Analysis in Metric Spaces with Bounded Curvature

Subgradient Projection Algorithms for Convex Feasibility on Riemannian Manifolds with Lower Bounded Curvatures

Computing the Gromov--Hausdorff distance using gradient methods

A splitting primal-dual proximity algorithm for solving composite optimization problems

A Descent Method for Nonsmooth Multiobjective Optimization in Hilbert Spaces

Anisotropic Proximal Gradient

Halpern inertial subgradient extragradient algorithm for solving equilibrium problems in Banach spaces

Convex optimization on CAT(0) cubical complexes

Subgradient ellipsoid method for nonsmooth convex problems

Dual Space Preconditioning for Gradient Descent

Derivatives with respect to metrics and applications: subgradient marching algorithm

A descent subgradient method using Mifflin line search for nonsmooth nonconvex optimization

A Refined Proximal Algorithm for Nonconvex Multiobjective Optimization in Hilbert Spaces

An improved inertial extragradient subgradient method for solving split variational inequality problems

A Nonlinear Bregman Primal-Dual Framework for Optimizing Nonconvex Infimal Convolutions

A Mirror Inertial Forward–Reflected–Backward Splitting: Convergence Analysis Beyond Convexity and Lipschitz Smoothness

Accelerated Primal-Dual Proximal Gradient Splitting Methods for Convex-Concave Saddle-Point Problems