Abstract:The least absolute shrinkage and selection operator (Lasso) is widely recognized across various fields of mathematics and engineering. Its variant, the generalized Lasso, finds extensive application in the fields of statistics, machine learning, image science, and related areas. Among the optimization techniques used to tackle this issue, saddle-point methods stand out, with the primal-dual hybrid gradient (PDHG) algorithm emerging as a particularly popular choice. However, the iterative behavior of PDHG remains poorly understood. In this paper, we employ dimensional analysis to derive a system of high-resolution ordinary differential equations (ODEs) tailored for PDHG. This system effectively captures a key feature of PDHG, the coupled $x$-correction and $y$-correction, distinguishing it from the proximal Arrow-Hurwicz algorithm. The small but essential perturbation ensures that PDHG consistently converges, bypassing the periodic behavior observed in the proximal Arrow-Hurwicz algorithm. Through Lyapunov analysis, We investigate the convergence behavior of the system of high-resolution ODEs and extend our insights to the discrete PDHG algorithm. Our analysis indicates that numerical errors resulting from the implicit scheme serve as a crucial factor affecting the convergence rate and monotonicity of PDHG, showcasing a noteworthy pattern also observed for the Alternating Direction Method of Multipliers (ADMM), as identified in [Li and Shi, 2024]. In addition, we further discover that when one component of the objective function is strongly convex, the iterative average of PDHG converges strongly at a rate $O(1/N)$, where $N$ is the number of iterations.

A first-order computational algorithm for reaction-diffusion type equations via primal-dual hybrid gradient method

High Accuracy Non-Equidistant Method for Singular Perturbation Reaction-Diffusion Problem

Boundary Stabilization of a Class of Reaction–advection–diffusion Systems Via a Gradient-Based Optimization Approach

A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations

Primal-dual hybrid gradient algorithms for computing time-implicit Hamilton-Jacobi equations

Acceleration of Primal–Dual Methods by Preconditioning and Simple Subproblem Procedures

Optimization of reaction parameters based on rSQP and hybrid automatic differentiation algorithm

A novel fourth-order scheme for two-dimensional Riesz space fractional nonlinear reaction-diffusion equations and its optimal preconditioned solver

A Primal-dual hybrid gradient method for solving optimal control problems and the corresponding Hamilton-Jacobi PDEs

Understanding the PDHG Algorithm via High-Resolution Differential Equations

A Sixth-order Finite Difference Method for the Two-dimensional Nonlinear Advection Diffusion Reaction Equation

A Parallel Algorithm for the Riesz Fractional Reaction-Diffusion Equation with Explicit Finite Difference Method

A Parallel iterative Algorithm for primal-dual weak Galerkin Schemes

Fast numerical algorithm for the reaction-diffusion equations using an interpolating method

A high order numerical scheme for solving a class of non‐homogeneous time‐fractional reaction diffusion equation

Restarted Primal-Dual Hybrid Conjugate Gradient Method for Large-Scale Quadratic Programming

Efficient bound preserving and asymptotic preserving semi-implicit schemes for the fast reaction-diffusion system

Parallel Primal-Dual Active-Set Algorithm with Nonlinear and Linear Preconditioners

Fast numerical schemes for nonlinear space-fractional multidelay reaction-diffusion equations by implicit integration factor methods

Robust hybrid finite element methods for reaction-dominated diffusion problems

Efficient Numerical Algorithms for Three-Dimensional Fractional Partial Differential Equations