Abstract:In this paper, we present novel yet simple homotopy proximal mapping algorithms for reconstructing a sparse signal from (noisy) linear measurements of the signal or for learning a sparse linear model from observed data, where the former task is well-known in the field of compressive sensing and the latter task is known as model selection in statistics and machine learning. The algorithms adopt a simple proximal mapping of the \(\ell _1\) norm at each iteration and gradually reduces the regularization parameter for the \(\ell _1\) norm. We prove a global linear convergence of the proposed homotopy proximal mapping (HPM) algorithms for recovering the sparse signal under three different settings (i) sparse signal recovery under noiseless measurements, (ii) sparse signal recovery under noisy measurements, and (iii) nearly-sparse signal recovery under sub-Gaussian noisy measurements. In particular, we show that when the measurement matrix satisfies restricted isometric properties (RIP), one of the proposed algorithms with an appropriate setting of a parameter based on the RIP constants converges linearly to the optimal solution up to the noise level. In addition, in setting (iii), a practical variant of the proposed algorithms does not rely on the RIP constants and our results for sparse signal recovery are better than the previous results in the sense that our recovery error bound is smaller. Furthermore, our analysis explicitly exhibits that more observations lead to not only more accurate recovery but also faster convergence. Finally our empirical studies provide further support for the proposed homotopy proximal mapping algorithm and verify the theoretical results.

A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

An Efficient Proximal Block Coordinate Homotopy Method for Large-Scale Sparse Least Squares Problems.

A Simple Homotopy Proximal Mapping Algorithm for Compressive Sensing

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Scaled Proximal Gradient Methods for Sparse Optimization Problems

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Homotopy Based Algorithms for $\ell _{\scriptscriptstyle 0}$ -Regularized Least-Squares

A Low-rank Projected Proximal Gradient Method for Spectral Compressed Sensing

An Iteratively Approximated Gradient Projection Algorithm for Sparse Signal Reconstruction

A Convergent Fixed-Point Proximity Algorithm Accelerated by FISTA for the ℓ0 Sparse Recovery Problem

Parameterized proximal-gradient algorithms for L1/L2 sparse signal recovery

Iteratively weighted thresholding homotopy method for the sparse solution of underdetermined linear equations

Fast Sparse Recovery Via Non-Convex Optimization

A Modified Proximal Gradient Method for a Class of Sparse Optimization Problems

An Efficient Semismooth Newton Method for Adaptive Sparse Signal Recovery Problems

On the Linear Convergence of a Proximal Gradient Method for a Class of Nonsmooth Convex Minimization Problems

A Non-convex Approach for Sparse Recovery with Convergence Guarantee

Linear Convergence of Stochastic Iterative Greedy Algorithms With Sparse Constraints

Time-Data Tradeoffs in Structured Signals Recovery Via the Proximal-Gradient Homotopy Method

An Alternating Manifold Proximal Gradient Method for Sparse Principal Component Analysis and Sparse Canonical Correlation Analysis