Abstract:Sparse signal restoration is usually formulated as the minimization of a quadratic cost function vertical bar vertical bar y - Ax vertical bar vertical bar(2)(2) where A is a dictionary and x is an unknown sparse vector. It is well-known that imposing an l(0) constraint leads to an NP-hard minimization problem. The convex relaxation approach has received considerable attention, where the to-norm is replaced by the l(1)-norm. Among the many effective l(1) solvers, the homotopy algorithm minimizes vertical bar vertical bar y - Ax vertical bar vertical bar(2)(2) + lambda vertical bar vertical bar x vertical bar(1) with respect to x for a continuum of lambda's. It is inspired by the piecewise regularity of the l(1)-regularization path, also referred to as the homotopy path. In this paper, we address the minimization problem vertical bar vertical bar y - Ax vertical bar vertical bar(2)(2) + lambda vertical bar vertical bar x vertical bar(0) for a continuum of lambda's and propose two heuristic search algorithms for l(0)-homotopy. Continuation Single Best Replacement is a forward backward greedy strategy extending the Single Best Replacement algorithm, previously proposed for to-minimization at a given lambda. The adaptive search of the lambda-values is inspired by l(1)-homotopy. l(0) Regularization Path Descent is a more complex algorithm exploiting the structural properties of the l(0)-regularization path, which is piecewise constant with respect to lambda. Both algorithms are empirically evaluated for difficult inverse problems involving ill-conditioned dictionaries. Finally, we show that they can be easily coupled with usual methods of model order selection.

Dual Extrapolation for Faster Lasso Solvers

Mind the duality gap: safer rules for the Lasso

Screening Rules for Lasso with Non-Convex Sparse Regularizers

Dual feature reduction for the sparse-group lasso and its adaptive variant

Simultaneous Inference for High-Dimensional Linear Models

A Safe Reinforced Feature Screening Strategy for Lasso Based on Feasible Solutions

Scaling Up Differentially Private LASSO Regularized Logistic Regression via Faster Frank-Wolfe Iterations

Fast Screening Rules for Optimal Design via Quadratic Lasso Reformulation

Safe Feature Elimination for the LASSO and Sparse Supervised Learning Problems

Extracting Dual Solutions via Primal Optimizers

A Fast Method for Lasso and Logistic Lasso

Proximal Iteration for Nonlinear Adaptive Lasso

A constructive approach to L

Lasso with Latents: Efficient Estimation, Covariate Rescaling, and Computational-Statistical Gaps

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

Accelerated Primal-Dual Proximal Gradient Splitting Methods for Convex-Concave Saddle-Point Problems

A Fast Solution to the Lasso Problem with Equality Constraints

Expanding boundaries of Gap Safe screening

Reduced-Space Iteratively Reweighted Second-Order Methods for Nonconvex Sparse Regularization

Homotopy Based Algorithms for $\ell _{\scriptscriptstyle 0}$ -Regularized Least-Squares

Look-Ahead Screening Rules for the Lasso