Abstract:In this paper, we revisit the iterative shrinkage–thresholding algorithm (ISTA) for solving linear inverse problems with sparse representation, commonly found in signal and image processing. We conduct numerical experiments and observe that the convergence behavior of ISTA tends to be linear instead of logarithmic, resulting in a flat approximation on the logarithmic-scale ordinate. Through meticulous observations, we find that assuming the smooth part to be strongly convex, rather than just convex, is more reasonable for the least-square model, even when dealing with ill-conditioned image matrices. We also improve the pivotal inequality for composite optimization by considering the smooth part to be strongly convex instead of general convex, which was first introduced in Li et al. (Proximal Subgradient Norm Minimization of ISTA and FISTA, arXiv:2211.01610, 2022). By utilizing this pivotal inequality, we extend linear convergence to composite optimization, both in terms of the objective value and the square of the proximal subgradient norm. To support our findings, we use a simple ill-conditioned matrix that allows for easy computation of the singular values, instead of the original blur matrix. The new numerical experiment demonstrates that the fast iterative shrinkage–thresholding algorithm for strongly convex functions has a faster linear convergence rate compared to ISTA. Furthermore, based on the tighter pivotal inequality, we also generalize the faster linear convergence rate to composite optimization, considering both the objective value and the square of the proximal subgradient norm. We achieve this by leveraging a well-constructed Lyapunov function with slight modifications and the phase-space representation based on the high-resolution ordinary differential equation framework from the implicit-velocity scheme.

On the convergence of the iterates of proximal gradient algorithm with extrapolation for convex nonsmooth minimization problems

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems

Global convergence of block proximal iteratively reweighted algorithm with extrapolation

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

A New Proximal Iterative Hard Thresholding Method with Extrapolation for $$\ell _0$$ Minimization

Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

On the convergence of an inertial proximal algorithm with a Tikhonov regularization term

Linear Convergence of ISTA and FISTA

A proximal-gradient inertial algorithm with Tikhonov regularization: strong convergence to the minimal norm solution

A Proximal-Gradient Method for Constrained Optimization

An Inexact Riemannian Proximal Gradient Method

Proximal Subgradient Norm Minimization of ISTA and FISTA

Proximal Point Algorithms with Inertial Extrapolation for Quasi-convex Pseudo-monotone Equilibrium Problems

Iteratively reweighted $\ell_1$ algorithms with extrapolation