Barzilai–Borwein-like rules in proximal gradient schemes for l1-regularized problems
Serena Crisci,Simone Rebegoldi,Gerardo Toraldo,Marco Viola
DOI: https://doi.org/10.1080/10556788.2023.2285489
2024-01-26
Optimization Methods and Software
Abstract:We propose a novel steplength selection rule in proximal gradient methods for minimizing the sum of a differentiable function plus an l1 -norm penalty term. The proposed rule modifies one of the classical Barzilai–Borwein steplength, extending analogous results obtained in the context of gradient projection methods for constrained optimization. We analyse the spectral properties of the Barzilai–Borwein-like steplength when the differentiable part is quadratic, showing that its reciprocal lies in the spectrum of the submatrix of the Hessian that depends on both the nonzero and the nonoptimal zero components of the current iterate, allowing for acceleration effects when the optimal zero components start to be identified. Furthermore, we insert the modified rule into a proximal gradient method with a nonmonotone line search, for which we prove global convergence towards a stationary point. Numerical experiments show the ability of the proposed rule to sweep the spectrum of the reduced Hessian on a series of quadratic l1 -regularized problems, as well as its effectiveness in recovering the ground truth in a least squares regularized problem arising in image restoration.
operations research & management science,mathematics, applied,computer science, software engineering