Abstract:It is well known that the minimum $\ell_2$-norm solution of the convex LASSO model, say $\mathbf{x}_{\star}$, is a continuous piecewise linear function of the regularization parameter $\lambda$, and its signed sparsity pattern is constant within each linear piece. The current study is an extension of this classic result, proving that the aforementioned properties extend to the min-norm solution map $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, where $\mathbf{y}$ is the observed signal, for a generalization of LASSO termed the scaled generalized minimax concave (sGMC) model. The sGMC model adopts a nonconvex debiased variant of the $\ell_1$-norm as sparse regularizer, but its objective function is overall-convex. Based on the geometric properties of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, we propose an extension of the least angle regression (LARS) algorithm, which iteratively computes the closed-form expression of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$ in each linear zone. Under suitable conditions, the proposed algorithm provably obtains the whole solution map $\mathbf{x}_{\star}(\mathbf{y},\lambda)$ within finite iterations. Notably, our proof techniques for establishing continuity and piecewise linearity of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$ are novel, and they lead to two side contributions: (a) our proofs establish continuity of the sGMC solution set as a set-valued mapping of $(\mathbf{y},\lambda)$; (b) to prove piecewise linearity and piecewise constant sparsity pattern of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, we do not require any assumption that previous work relies on (whereas to prove some additional properties of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, we use a different set of assumptions from previous work).

On the Hardness of the $L_1-L_2$ Regularization Problem

On NP-Hardness of $L_1/L_2$ Minimization and Bound Theory of Nonzero Entries in Solutions

&Lt;formula Formulatype="inline"> <tex Notation="TeX">$L_{1/2}$</tex></formula> Regularization: A Thresholding Representation Theory and a Fast Solver

Fast Learning with Nonconvex L1-2 Regularization.

Sparse Deep Learning Models with the $\ell_1$ Regularization

On recovery of sparse signals via l1 minimization

Study on <i>L</i><sub>1</sub> over <i>L</i><sub>2</sub> Minimization for Nonnegative Signal Recovery

Regularization for Compressive Sensing

ℓ _1/2,1 group sparse regularization for compressive sensing

$L_{1/2}$ Regularization: Convergence of Iterative Half Thresholding Algorithm

Minimizing L 1 over L 2 norms on the gradient

Lower Bound Theory of Nonzero Entries in Solutions of $\Ell_2$-$\Ell_p$ Minimization

Sparsity via Sparse Group $k$-max Regularization

A Unified Study on $L_1$ over $L_2$ Minimization

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Solving the L1 regularized least square problem via a box-constrained smooth minimization

A Simple Neural Network for Sparse Optimization with $l_1$ Regularization

Piecewise Linearity of Min-Norm Solution Map of a Nonconvexly Regularized Convex Sparse Model

l1-norm regularized l1-norm best-fit lines

Study on L_1 over L_2 Minimization for Nonnegative Signal Recovery

Robust Sparse Analysis Regularization