Abstract:In this paper, we revisit the iterative shrinkage–thresholding algorithm (ISTA) for solving linear inverse problems with sparse representation, commonly found in signal and image processing. We conduct numerical experiments and observe that the convergence behavior of ISTA tends to be linear instead of logarithmic, resulting in a flat approximation on the logarithmic-scale ordinate. Through meticulous observations, we find that assuming the smooth part to be strongly convex, rather than just convex, is more reasonable for the least-square model, even when dealing with ill-conditioned image matrices. We also improve the pivotal inequality for composite optimization by considering the smooth part to be strongly convex instead of general convex, which was first introduced in Li et al. (Proximal Subgradient Norm Minimization of ISTA and FISTA, arXiv:2211.01610, 2022). By utilizing this pivotal inequality, we extend linear convergence to composite optimization, both in terms of the objective value and the square of the proximal subgradient norm. To support our findings, we use a simple ill-conditioned matrix that allows for easy computation of the singular values, instead of the original blur matrix. The new numerical experiment demonstrates that the fast iterative shrinkage–thresholding algorithm for strongly convex functions has a faster linear convergence rate compared to ISTA. Furthermore, based on the tighter pivotal inequality, we also generalize the faster linear convergence rate to composite optimization, considering both the objective value and the square of the proximal subgradient norm. We achieve this by leveraging a well-constructed Lyapunov function with slight modifications and the phase-space representation based on the high-resolution ordinary differential equation framework from the implicit-velocity scheme.

The Impact of Data Dependence, Convergence and Stability by $AT$ Iterative Algorithms

Adaptive Iterative Hard Thresholding for Least Absolute Deviation Problems with Sparsity Constraints

On the Convergence of the Accelerated Riccati Iteration Method.

A New Linear Convergence Result for the Iterative Soft Thresholding Algorithm

Disquisition on convergence, stability, and data dependence for a new fast iterative process

Strong Convergence of FISTA Iterates under H{ö}lderian and Quadratic Growth Conditions

A robust alternative to examine data dependency of fixed points of quasi-contractive operators: an efficient approach that relies on the collage theorem

Convergence of algorithms for fixed points of relatively nonexpansive mappings via Ishikawa iteration

Convergence of Algorithms for Fixed Points of Generalized Asymptotically Quasi- Φ -Nonexpansive Mappings with Applications

A Strongly Convergent Algorithm for Approximating a Common Solution to Fixed Point and Mixed Equilibrium Problems

Stability of the F∗ Algorithm on Strong Pseudocontractive Mapping and Its Application

An Inertial Mann-like Algorithm for Fixed Points of Nonexpansive Mappings in Hilbert Spaces

A Krasnoselskii–Ishikawa Iterative Algorithm for Monotone Reich Contractions in Partially Ordered Banach Spaces with an Application

Iterative Methods for Zero Points of Accretive Operators in Banach Spaces

A New Iterative Algorithm for Equilibrium and Fixed Point Problems of Nonexpansive Mapping

Rates of convergence for iterative solutions of equations involving set-valued accretive operators

A General Implicit Iteration for Finding Fixed Points of Nonexpansive Mappings

Linear Convergence Rates for Extrapolated Fixed Point Algorithms

Linear Convergence of ISTA and FISTA

Generalized Mann Iterations for Approximating Fixed Points of a Family of Hemicontractions

Modified Inertial Projection and Contraction Algorithms with Non-Monotonic Step Sizes for Solving Variational Inequalities and Their Applications