Abstract:The generalized Lasso is a remarkably versatile and extensively utilized model across a broad spectrum of domains, including statistics, machine learning, and image science. Among the optimization techniques employed to address the challenges posed by this model, saddle-point methods stand out for their effectiveness. In particular, the primal-dual hybrid gradient (PDHG) algorithm has emerged as a highly popular choice, celebrated for its robustness and efficiency in finding optimal solutions. Recently, the underlying mechanism of the PDHG algorithm has been elucidated through the high-resolution ordinary differential equation (ODE) and the implicit-Euler scheme as detailed in [Li and Shi, 2024a]. This insight has spurred the development of several accelerated variants of the PDHG algorithm, originally proposed by [Chambolle and Pock, 2011]. By employing discrete Lyapunov analysis, we establish that the PDHG algorithm with iteration-varying step sizes, converges at a rate near $O(1/k^2)$. Furthermore, for the specific setting where $\tau_{k+1}\sigma_k = s^2$ and $\theta_k = \tau_{k+1}/\tau_k \in (0, 1)$ as proposed in [Chambolle and Pock, 2011], an even faster convergence rate of $O(1/k^2)$ can be achieved. To substantiate these findings, we design a novel discrete Lyapunov function. This function is distinguished by its succinctness and straightforwardness, providing a clear and elegant proof of the enhanced convergence properties of the PDHG algorithm under the specified conditions. Finally, we utilize the discrete Lyapunov function to establish the optimal linear convergence rate when both the objective functions are strongly convex.

Exponential Stability of Partial Primal-Dual Gradient Dynamics with Nonsmooth Objective Functions

Boundary Stabilization of a Class of Reaction–advection–diffusion Systems Via a Gradient-Based Optimization Approach

Exponential Stabilization of Unstable Diffusion-Reaction Pdes with Mixed Boundary Condition

Exponential Stabilization of an Unstable Parabolic PDE System Using Boundary Optimal Control

Output stabilization of boundary-controlled parabolic PDEs via gradient-based dynamic optimization

Stability of Primal-Dual Gradient Flow Dynamics for Multi-Block Convex Optimization Problems

Proximal Gradient Dynamics: Monotonicity, Exponential Convergence, and Applications

Exponential Convergence of Augmented Primal-dual Gradient Algorithms for Partially Strongly Convex Functions

An SDE Perspective on Stochastic Inertial Gradient Dynamics with Time-Dependent Viscosity and Geometric Damping

Primal Dual Alternating Proximal Gradient Algorithms for Nonsmooth Nonconvex Minimax Problems with Coupled Linear Constraints

A dynamic gradient approach to Pareto optimization with nonsmooth convex objective functions

Accelerated Primal-Dual Proximal Gradient Splitting Methods for Convex-Concave Saddle-Point Problems

A Second Order Primal–Dual Dynamical System for a Convex–Concave Bilinear Saddle Point Problem

Tikhonov regularized mixed-order primal-dual dynamical system for convex optimization problems with linear equality constraints

On Stochastic Primal-Dual Hybrid Gradient Approach for Compositely Regularized Minimization.

A Lyapunov Analysis of Accelerated PDHG Algorithms

Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems

Fast Convergence of Inertial Multiobjective Gradient-like Systems with Asymptotic Vanishing Damping

Primal-dual hybrid gradient algorithms for computing time-implicit Hamilton-Jacobi equations

Strong asymptotic convergence of a slowly damped inertial primal-dual dynamical system controlled by a Tikhonov regularization term