Abstract:It is well known that the minimum $\ell_2$-norm solution of the convex LASSO model, say $\mathbf{x}_{\star}$, is a continuous piecewise linear function of the regularization parameter $\lambda$, and its signed sparsity pattern is constant within each linear piece. The current study is an extension of this classic result, proving that the aforementioned properties extend to the min-norm solution map $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, where $\mathbf{y}$ is the observed signal, for a generalization of LASSO termed the scaled generalized minimax concave (sGMC) model. The sGMC model adopts a nonconvex debiased variant of the $\ell_1$-norm as sparse regularizer, but its objective function is overall-convex. Based on the geometric properties of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, we propose an extension of the least angle regression (LARS) algorithm, which iteratively computes the closed-form expression of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$ in each linear zone. Under suitable conditions, the proposed algorithm provably obtains the whole solution map $\mathbf{x}_{\star}(\mathbf{y},\lambda)$ within finite iterations. Notably, our proof techniques for establishing continuity and piecewise linearity of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$ are novel, and they lead to two side contributions: (a) our proofs establish continuity of the sGMC solution set as a set-valued mapping of $(\mathbf{y},\lambda)$; (b) to prove piecewise linearity and piecewise constant sparsity pattern of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, we do not require any assumption that previous work relies on (whereas to prove some additional properties of $\mathbf{x}_{\star}(\mathbf{y},\lambda)$, we use a different set of assumptions from previous work).

Piecewise Linear Relationship Between L1 Norm Objective Functional Values and L∞ Norm Constraint Specifications

An Improved Smoothing Technique-Based Control Vector Parameterization Method for Optimal Control Problems with Inequality Path Constraints

A Novel Penalty Approach for Nonlinear Dynamic Optimization Problems with Inequality Path Constraints

Relative Regularity Conditions and Linear Regularity Properties for Split Feasibility Problems in Normed Linear Spaces

A Sufficient Condition on Monotonic Increase of the Number of Nonzero Entry in the Optimizer of L1 Norm Penalized Least-Square Problem

A Simple Neural Network for Sparse Optimization with $l_1$ Regularization

Linearly Constrained Global Optimization Via Piecewise-Linear Approximation

Optimality analysis on l0/1 -sparsity regularized nonsmooth optimization

Revisiting $L_q(0\leq q<1)$ Norm Regularized Optimization

l1-norm regularized l1-norm best-fit lines

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Neural Network for a Class of Sparse Optimization with L0-regularization.

Linear Programming-Based Sparse Kernel Regression with L1-Norm Minimization for Nonlinear System Modeling

Alternative LMI characterizations for fractional-order linear systems

Piecewise Linearity of Min-Norm Solution Map of a Nonconvexly Regularized Convex Sparse Model

Group Sparse Optimization Via Lp, Q Regularization.

Identification and Optimization of Lattice Piecewise-Linear Functions

Approximation and Optimization of L1-norm for Continuous-Time Linear Systems

Lower Bound Theory of Nonzero Entries in Solutions of $\Ell_2$-$\Ell_p$ Minimization

On NP-Hardness of $L_1/L_2$ Minimization and Bound Theory of Nonzero Entries in Solutions

Solving of L0 Norm Constrained EEG Inverse Problem.