Abstract:In this paper, we revisit the iterative shrinkage–thresholding algorithm (ISTA) for solving linear inverse problems with sparse representation, commonly found in signal and image processing. We conduct numerical experiments and observe that the convergence behavior of ISTA tends to be linear instead of logarithmic, resulting in a flat approximation on the logarithmic-scale ordinate. Through meticulous observations, we find that assuming the smooth part to be strongly convex, rather than just convex, is more reasonable for the least-square model, even when dealing with ill-conditioned image matrices. We also improve the pivotal inequality for composite optimization by considering the smooth part to be strongly convex instead of general convex, which was first introduced in Li et al. (Proximal Subgradient Norm Minimization of ISTA and FISTA, arXiv:2211.01610, 2022). By utilizing this pivotal inequality, we extend linear convergence to composite optimization, both in terms of the objective value and the square of the proximal subgradient norm. To support our findings, we use a simple ill-conditioned matrix that allows for easy computation of the singular values, instead of the original blur matrix. The new numerical experiment demonstrates that the fast iterative shrinkage–thresholding algorithm for strongly convex functions has a faster linear convergence rate compared to ISTA. Furthermore, based on the tighter pivotal inequality, we also generalize the faster linear convergence rate to composite optimization, considering both the objective value and the square of the proximal subgradient norm. We achieve this by leveraging a well-constructed Lyapunov function with slight modifications and the phase-space representation based on the high-resolution ordinary differential equation framework from the implicit-velocity scheme.

A New Linear Convergence Result for the Iterative Soft Thresholding Algorithm

Adaptive Iterative Hard Thresholding for Least Absolute Deviation Problems with Sparsity Constraints

Linear Convergence of ISTA and FISTA

Local Linear Convergence of ISTA and FISTA on the LASSO Problem

One-Bit Compressive Sensing with Projected Subgradient Method under Sparsity Constraints

CONVERGENCE AND STABILITY OF ITERATIVELY REWEIGHTED LEAST SQUARES FOR LOW-RANK MATRIXRECOVERY

Linear convergence of adaptively iterative thresholding algorithms for compressed sensing

Strong Convergence of FISTA Iterates under H{ö}lderian and Quadratic Growth Conditions

A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

The Convergence of Least-Squares Progressive Iterative Approximation for Singular Least-Squares Fitting System.

Hybrid ISTA: Unfolding ISTA With Convergence Guarantees Using Free-Form Deep Neural Networks

Compressed Lista Exploiting Toeplitz Structure

Learned Extragradient ISTA with Interpretable Residual Structures for Sparse Coding.

A new inexact iterative hard thresholding algorithm for compressed sensing

Sparse solution of underdetermined linear equations via adaptively iterative thresholding

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding

A Guaranteed Convergence Analysis for the Projected Fast Iterative Soft-Thresholding Algorithm in Parallel MRI

Stability and Risk Bounds of Iterative Hard Thresholding

Fast Iterative Hard Thresholding for Compressed Sensing

Constrained Mix Sparse Optimization via Hard Thresholding Pursuit