Abstract:We design, analyze and test a golden ratio primal–dual algorithm (GRPDA) for solving structured convex optimization problem, where the objective function is the sum of two closed proper convex functions, one of which involves a composition with a linear transform. GRPDA preserves all the favorable features of the classical primal–dual algorithm (PDA), i.e., the primal and the dual variables are updated in a Gauss–Seidel manner, and the per iteration cost is dominated by the evaluation of the proximal point mappings of the two component functions and two matrix-vector multiplications. Compared with the classical PDA, which takes an extrapolation step, the novelty of GRPDA is that it is constructed based on a convex combination of essentially the whole iteration trajectory. We show that GRPDA converges within a broader range of parameters than the classical PDA, provided that the reciprocal of the convex combination parameter is bounded above by the golden ratio, which explains the name of the algorithm. An 𝒪(1/N) ergodic convergence rate result is also established based on the primal–dual gap function, where N denotes the number of iterations. When either the primal or the dual problem is strongly convex, an accelerated GRPDA is constructed to improve the ergodic convergence rate from 𝒪(1/N) to 𝒪(1/N^2) . Moreover, we show for regularized least-squares and linear equality constrained problems that the reciprocal of the convex combination parameter can be extended from the golden ratio to 2 and meanwhile a relaxation step can be taken. Our preliminary numerical results on LASSO, nonnegative least-squares and minimax matrix game problems, with comparisons to some state-of-the-art relative algorithms, demonstrate the efficiency of the proposed algorithms.

New convergence results of the golden ratio primal-dual algorithm

A Golden Ratio Primal–Dual Algorithm for Structured Convex Optimization

GRPDA Revisited: Relaxed Condition and Connection to Chambolle-Pock's Primal-Dual Algorithm.

A new primal-dual algorithm for structured convex optimization involving a lipschitzian term

An inexact golden ratio primal-dual algorithm with linesearch step for a saddle point problem

Preconditioned golden ratio primal-dual algorithm with linesearch

Accelerated Primal-Dual Proximal Gradient Splitting Methods for Convex-Concave Saddle-Point Problems

A Golden Ratio Proximal Alternating Direction Method of Multipliers for Separable Convex Optimization

A Unified Primal-Dual Algorithm Framework for Inequality Constrained Problems

Stochastic Primal-Dual Algorithms with Faster Convergence Than O(1/√T) for Problems Without Bilinear Structure

convergence rates of (dual-primal) balanced augmented Lagrangian methods for linearly constrained convex programming

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

A convex combination based primal-dual algorithm with linesearch for general convex-concave saddle point problems

Stochastic Primal-Dual Algorithms with Faster Convergence than $O(1/\sqrt{T})$ for Problems without Bilinear Structure

Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization

On the Geometry and Refined Rate of Primal-Dual Hybrid Gradient for Linear Programming

Golden Ratio Proximal Gradient ADMM for Distributed Composite Convex Optimization

A splitting preconditioned primal-dual algorithm with interpolation and extrapolation for bilinear saddle point problem

Monitoring the Convergence Speed of PDHG to Find Better Primal and Dual Step Sizes

New extrapolation projection contraction algorithms based on the golden ratio for pseudo-monotone variational inequalities

Auto-conditioned primal-dual hybrid gradient method and alternating direction method of multipliers