Abstract:Iteratively Re-weighted Least Squares (IRLS) is a method for solving minimization problems involving non-quadratic cost functions, perhaps non-convex and non-smooth, which however can be described as the infimum over a family of quadratic functions. This transformation suggests an algorithmic scheme that solves a sequence of quadratic problems to be tackled efficiently by tools of numerical linear algebra. Its general scope and its usually simple implementation, transforming the initial non-convex and non-smooth minimization problem into a more familiar and easily solvable quadratic optimization problem, make it a versatile algorithm. However, despite its simplicity, versatility, and elegant analysis, the complexity of IRLS strongly depends on the way the solution of the successive quadratic optimizations is addressed. For the important special case of $\textit{compressed sensing}$ and sparse recovery problems in signal processing, we investigate theoretically and numerically how accurately one needs to solve the quadratic problems by means of the $\textit{conjugate gradient}$ (CG) method in each iteration in order to guarantee convergence. The use of the CG method may significantly speed-up the numerical solution of the quadratic subproblems, in particular, when fast matrix-vector multiplication (exploiting for instance the FFT) is available for the matrix involved. In addition, we study convergence rates. Our modified IRLS method outperforms state of the art first order methods such as Iterative Hard Thresholding (IHT) or Fast Iterative Soft-Thresholding Algorithm (FISTA) in many situations, especially in large dimensions. Moreover, IRLS is often able to recover sparse vectors from fewer measurements than required for IHT and FISTA.

Fast Iterative Reweighted Least Squares Algorithm for Sparse Signals Recovery.

CONVERGENCE AND STABILITY OF ITERATIVELY REWEIGHTED LEAST SQUARES FOR LOW-RANK MATRIXRECOVERY

Fast Iteratively Reweighted Least Squares Minimization for Sparse Recovery

Improved Iteratively Reweighted Least Squares Algorithms for Sparse Recovery Problem

Weighted LASSO for Sparse Recovery with Statistical Prior Support Information

Iteratively re-weighted least squares minimization for sparse recovery

Conjugate gradient acceleration of iteratively re-weighted least squares methods

Precise asymptotics of reweighted least-squares algorithms for linear diagonal networks

An Iteratively Reweighted Method For Recovery Of Block-Sparse Signal With Unknown Block Partition

Smoothed Low Rank and Sparse Matrix Recovery by Iteratively Reweighted Least Squares Minimization

A New Reweighted Algorithm with Support Detection for Compressed Sensing

Reweighted Linearized Bregman Algorithm for Sparse Signal Recovery

A Novel Algorithm for Compressive Sensing: Iteratively Reweighed Operator Algorithm (IROA)

Projective Iterative Hard Thresholding Algorithm for Sparse Signal Recovery

Convergence and stability of iteratively reweighted least squares for low-rank matrix recovery

Robust Recovery of Structured Sparse Signals with Uncertain Sensing Matrix: A Turbo-VBI Approach.

A Method of Reweighting the Sensing Matrix for Compressed Sensing

An Iterative Weighing Algorithm for Image Reconstruction in Compressive Sensing

Fast Recursive Greedy Methods for Sparse Signal Recovery

An efficient algorithm with fast convergence rate for sparse graph signal reconstruction

Learning Sparse and Low-Rank Priors for Image Recovery via Iterative Reweighted Least Squares Minimization