Abstract:Iteratively Re-weighted Least Squares (IRLS) is a method for solving minimization problems involving non-quadratic cost functions, perhaps non-convex and non-smooth, which however can be described as the infimum over a family of quadratic functions. This transformation suggests an algorithmic scheme that solves a sequence of quadratic problems to be tackled efficiently by tools of numerical linear algebra. Its general scope and its usually simple implementation, transforming the initial non-convex and non-smooth minimization problem into a more familiar and easily solvable quadratic optimization problem, make it a versatile algorithm. However, despite its simplicity, versatility, and elegant analysis, the complexity of IRLS strongly depends on the way the solution of the successive quadratic optimizations is addressed. For the important special case of $\textit{compressed sensing}$ and sparse recovery problems in signal processing, we investigate theoretically and numerically how accurately one needs to solve the quadratic problems by means of the $\textit{conjugate gradient}$ (CG) method in each iteration in order to guarantee convergence. The use of the CG method may significantly speed-up the numerical solution of the quadratic subproblems, in particular, when fast matrix-vector multiplication (exploiting for instance the FFT) is available for the matrix involved. In addition, we study convergence rates. Our modified IRLS method outperforms state of the art first order methods such as Iterative Hard Thresholding (IHT) or Fast Iterative Soft-Thresholding Algorithm (FISTA) in many situations, especially in large dimensions. Moreover, IRLS is often able to recover sparse vectors from fewer measurements than required for IHT and FISTA.

A modified PRP conjugate gradient method with inertial extrapolation for sparse signal reconstruction

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Convergence Analysis of Projected Gradient Descent for Schatten-p Nonconvex Matrix Recovery

A Modified Spectral Conjugate Gradient Projection Method for Signal Recovery.

Parameterized proximal-gradient algorithms for L1/L2 sparse signal recovery

Group Projected Subspace Pursuit for Block Sparse Signal Reconstruction: Convergence Analysis and Applications

Conjugate gradient acceleration of iteratively re-weighted least squares methods

Intelligent Greedy Pursuit Model for Sparse Reconstruction Based on L0 Minimization

Conjugate Gradient Hard Thresholding Pursuit Algorithm For Sparse Signal Recovery

A modified inertial three-term conjugate gradient projection method for constrained nonlinear equations with applications in compressed sensing

A Non-convex Approach for Sparse Recovery with Convergence Guarantee

An inertial Fletcher–Reeves-type conjugate gradient projection-based method and its spectral extension for constrained nonlinear equations

Gradient Projection with Approximate L0 Norm Minimization for Sparse Reconstruction in Compressed Sensing

Sparse Signal Reconstruction Via Expanded Subspace Pursuit

Proximal Iteratively Reweighted Algorithm with Multiple Splitting for Nonconvex Sparsity Optimization

From Least Squares to Sparse: A Non-Convex Approach with Guarantee

A Fast Inverse Polynomial Reconstruction Method Based on Conformal Fourier Transformation

A Modified Spectral PRP Conjugate Gradient Projection Method for Solving Large-Scale Monotone Equations and Its Application in Compressed Sensing

Projected Gradient Descent Algorithms for Solving Nonlinear Inverse Problems with Generative Priors

Efficient Projected Gradient Methods for L 0 Constrained Optimization

A Low-rank Projected Proximal Gradient Method for Spectral Compressed Sensing