Abstract:We design, analyze and test a golden ratio primal–dual algorithm (GRPDA) for solving structured convex optimization problem, where the objective function is the sum of two closed proper convex functions, one of which involves a composition with a linear transform. GRPDA preserves all the favorable features of the classical primal–dual algorithm (PDA), i.e., the primal and the dual variables are updated in a Gauss–Seidel manner, and the per iteration cost is dominated by the evaluation of the proximal point mappings of the two component functions and two matrix-vector multiplications. Compared with the classical PDA, which takes an extrapolation step, the novelty of GRPDA is that it is constructed based on a convex combination of essentially the whole iteration trajectory. We show that GRPDA converges within a broader range of parameters than the classical PDA, provided that the reciprocal of the convex combination parameter is bounded above by the golden ratio, which explains the name of the algorithm. An 𝒪(1/N) ergodic convergence rate result is also established based on the primal–dual gap function, where N denotes the number of iterations. When either the primal or the dual problem is strongly convex, an accelerated GRPDA is constructed to improve the ergodic convergence rate from 𝒪(1/N) to 𝒪(1/N^2) . Moreover, we show for regularized least-squares and linear equality constrained problems that the reciprocal of the convex combination parameter can be extended from the golden ratio to 2 and meanwhile a relaxation step can be taken. Our preliminary numerical results on LASSO, nonnegative least-squares and minimax matrix game problems, with comparisons to some state-of-the-art relative algorithms, demonstrate the efficiency of the proposed algorithms.

A New Primal-Dual Algorithm for a Class of Nonlinear Compositional Convex Optimization Problems

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

A splitting primal-dual proximity algorithm for solving composite optimization problems

A Smooth Primal-Dual Optimization Framework for Nonsmooth Composite Convex Minimization

A Nonlinear Bregman Primal-Dual Framework for Optimizing Nonconvex Infimal Convolutions

A Fast Single-Loop Primal-Dual Algorithm for Non-Convex Functional Constrained Optimization

Optimal Primal-Dual Algorithm with Last iterate Convergence Guarantees for Stochastic Convex Optimization Problems

Primal Dual Alternating Proximal Gradient Algorithms for Nonsmooth Nonconvex Minimax Problems with Coupled Linear Constraints

A new primal-dual algorithm for structured convex optimization involving a lipschitzian term

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

A fast primal-dual algorithm via dynamical system with variable mass for linearly constrained convex optimization

A Unified Algorithmic Framework for Distributed Composite Optimization.

A Unified Contraction Analysis of a Class of Distributed Algorithms for Composite Optimization

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

A Universal Accelerated Primal–Dual Method for Convex Optimization Problems

A Primal-Dual Smoothing Framework for Max-Structured Non-Convex Optimization

A primal-dual interior-point relaxation method with global and rapidly local convergence for nonlinear programs

An Augmented Lagrangian Primal-Dual Semismooth Newton Method for Multi-Block Composite Optimization

Accelerated Primal-dual Scheme for a Class of Stochastic Nonconvex-concave Saddle Point Problems

Convergence analysis of a primal-dual optimization-by-continuation algorithm

A Golden Ratio Primal–Dual Algorithm for Structured Convex Optimization