Abstract:Iteratively Re-weighted Least Squares (IRLS) is a method for solving minimization problems involving non-quadratic cost functions, perhaps non-convex and non-smooth, which however can be described as the infimum over a family of quadratic functions. This transformation suggests an algorithmic scheme that solves a sequence of quadratic problems to be tackled efficiently by tools of numerical linear algebra. Its general scope and its usually simple implementation, transforming the initial non-convex and non-smooth minimization problem into a more familiar and easily solvable quadratic optimization problem, make it a versatile algorithm. However, despite its simplicity, versatility, and elegant analysis, the complexity of IRLS strongly depends on the way the solution of the successive quadratic optimizations is addressed. For the important special case of $\textit{compressed sensing}$ and sparse recovery problems in signal processing, we investigate theoretically and numerically how accurately one needs to solve the quadratic problems by means of the $\textit{conjugate gradient}$ (CG) method in each iteration in order to guarantee convergence. The use of the CG method may significantly speed-up the numerical solution of the quadratic subproblems, in particular, when fast matrix-vector multiplication (exploiting for instance the FFT) is available for the matrix involved. In addition, we study convergence rates. Our modified IRLS method outperforms state of the art first order methods such as Iterative Hard Thresholding (IHT) or Fast Iterative Soft-Thresholding Algorithm (FISTA) in many situations, especially in large dimensions. Moreover, IRLS is often able to recover sparse vectors from fewer measurements than required for IHT and FISTA.

Fast RLS-type Algorithm for Unbiased Equation-Error Adaptive IIR Filtering Based on Approximate Inverse-Power Iteration

A recursive adaptive lattice filters for impulsive signals

General A-P Iterative Algorithm in Shift-invariant Spaces

A Robust Adaptive FIR Filter Based on M-Estimation

Adaptive Iterative Learning Control for Discrete‐time Nonlinear Systems with Multiple Iteration‐varying High‐order Internal Models

A Fast Proportionate RLS Adaptive Equalization for Underwater Acoustic Communications

Lattice Form Adaptive Infinite Impulse Response Filtering Algorithm for Active Noise Control.

Gauss-Seidel Progressive and Iterative Approximation for Least Squares Fitting

Adaptive Iterative Learning Control Mechanism for Nonlinear Systems Subject to High-Order Internal Model

An Efficient Iterative Algorithm for Accurately Calculating Impulse Response Functions

A High-Performance and Energy-Efficient FIR Adaptive Filter Using Approximate Distributed Arithmetic Circuits

Fast Recursive Least Squares Adaptive Filtering by Fast Fourier Transform-Based Conjugate Gradient Iterations

Accelerated Convergence Interleaving Iterative Learning Control and Inverse Dynamics Identification

Conjugate gradient acceleration of iteratively re-weighted least squares methods

A New Adaptive IIR Algorithm for Active Noise Control.

A Novel Adaptive Filter Algorithm Based on Dfp Technique

Discrete Time $q$-Lag Maximum Likelihood FIR Smoothing and Iterative Recursive Algorithm

Constructing numerically stable Kalman filter-based algorithms for gradient-based adaptive filtering

Adaptive operator learning for infinite-dimensional Bayesian inverse problems

Fast Iterative Reweighted Least Squares Algorithm for Sparse Signals Recovery.

Approximate Optimal Filter for Linear Gaussian Time-invariant Systems