Abstract:The ℓ-1 norm based optimization is widely used in signal processing, especially in recent compressed sensing theory. This paper studies the solution path of the ℓ-1 norm penalized least-square problem, whose constrained form is known as Least Absolute Shrinkage and Selection Operator (LASSO). A solution path is the set of all the optimizers with respect to the evolution of the hyperparameter (Lagrange multiplier). The study of the solution path is of great significance in viewing and understanding the profile of the tradeoff between the approximation and regularization terms. If the solution path of a given problem is known, it can help us to find the optimal hyperparameter under a given criterion such as the Akaike Information Criterion. In this paper we present a sufficient condition on ℓ-1 norm penalized least-square problem. Under this sufficient condition, the number of nonzero entries in the optimizer or solution vector increases monotonically when the hyperparameter decreases. We also generalize the result to the often used total variation case, where the ℓ-1 norm is taken over the first order derivative of the solution vector. We prove that the proposed condition has intrinsic connections with the condition given by Donoho, et al and the positive cone condition by Efron el al . However, the proposed condition does not need to assume the sparsity level of the signal as required by Donoho et al's condition, and is easier to verify than Efron, et al's positive cone condition when being used for practical applications.

Monotonic Properties of the Least Squares Mean

Monotonicities and Schur-Convexity for a Type of Symmetric Mean Values

Monotonicity in Quadratically Regularized Linear Programs

Weak Sharp Minima on Riemannian Manifolds

Alternative Proofs for Monotonic and Logarithmically Convex Properties of One-Parameter Mean Values

Monotone and Accretive Vector Fields on Riemannian Manifolds

A Note on the Sensitivity of the Solution of the Weighted Linear Least Squares Problem

A Note on the Scaled Total Least Squares Problem

Bounds for the Wasserstein mean with applications to the Lie-Trotter mean

Mean Field Games of Controls: Propagation of Monotonicities

A Sufficient Condition on Monotonic Increase of the Number of Nonzero Entry in the Optimizer of L1 Norm Penalized Least-Square Problem

Least Squares Properties of Generalized Inverses

Concavity property of minimal $L^{2}$ integrals with Lebesgue measurable gain VII -- Negligible weights

On monotonicity conditions for Mean Field Games

No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis

On NP-Hardness of $L_1/L_2$ Minimization and Bound Theory of Nonzero Entries in Solutions

A characterization of the Logarithmic Least Squares Method

The QLY Least-Squares and the QLY Least-Squares Minimal-Norm of Linear Dual Least Squares Problems

Least squares formulation for ill-posed inverse problems and applications

Bounded Solutions for Non-parametric Mean Curvature Problems with Nonlinear Terms

Fine properties of nonlinear potentials and a unified perspective on monotonicity formulas