Abstract:In this paper, we revisit the iterative shrinkage–thresholding algorithm (ISTA) for solving linear inverse problems with sparse representation, commonly found in signal and image processing. We conduct numerical experiments and observe that the convergence behavior of ISTA tends to be linear instead of logarithmic, resulting in a flat approximation on the logarithmic-scale ordinate. Through meticulous observations, we find that assuming the smooth part to be strongly convex, rather than just convex, is more reasonable for the least-square model, even when dealing with ill-conditioned image matrices. We also improve the pivotal inequality for composite optimization by considering the smooth part to be strongly convex instead of general convex, which was first introduced in Li et al. (Proximal Subgradient Norm Minimization of ISTA and FISTA, arXiv:2211.01610, 2022). By utilizing this pivotal inequality, we extend linear convergence to composite optimization, both in terms of the objective value and the square of the proximal subgradient norm. To support our findings, we use a simple ill-conditioned matrix that allows for easy computation of the singular values, instead of the original blur matrix. The new numerical experiment demonstrates that the fast iterative shrinkage–thresholding algorithm for strongly convex functions has a faster linear convergence rate compared to ISTA. Furthermore, based on the tighter pivotal inequality, we also generalize the faster linear convergence rate to composite optimization, considering both the objective value and the square of the proximal subgradient norm. We achieve this by leveraging a well-constructed Lyapunov function with slight modifications and the phase-space representation based on the high-resolution ordinary differential equation framework from the implicit-velocity scheme.

The variant of the iterative shrinkage-thresholding algorithm for minimization of the l 1 over l ∞ norms

Adaptive Iterative Hard Thresholding for Least Absolute Deviation Problems with Sparsity Constraints

One-Bit Compressive Sensing with Projected Subgradient Method under Sparsity Constraints

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Minimizing L 1 over L 2 norms on the gradient

Two-level ℓ1 Minimization for Compressed Sensing

Minimization of Transformed $L_1$ Penalty: Theory, Difference of Convex Function Algorithm, and Robust Application in Compressed Sensing

Linear Convergence of ISTA and FISTA

A New Linear Convergence Result for the Iterative Soft Thresholding Algorithm

On the Convergence of an Active-Set Method for ℓ1minimization

Iterative thresholding algorithm based on non-convex method for modified lp-norm regularization minimization

An Alternating Iterative Algorithm for Image Deblurring and Denoising Problems

A new inexact iterative hard thresholding algorithm for compressed sensing

Nonconvex Sorted ℓ _1 Minimization for Sparse Approximation

A wonderful triangle in compressed sensing

New Inexact Explicit Thresholding/shrinkage Formulas for Inverse Problems with Overlapping Group Sparsity.

Proximal Subgradient Norm Minimization of ISTA and FISTA

A Fast Algorithm for Sparse Reconstruction Based on Shrinkage, Subspace Optimization, and Continuation

Fast Sparse Recovery Via Non-Convex Optimization

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems

Iteratively re-weighted least squares minimization for sparse recovery