Abstract:The ℓ-1 norm based optimization is widely used in signal processing, especially in recent compressed sensing theory. This paper studies the solution path of the ℓ-1 norm penalized least-square problem, whose constrained form is known as Least Absolute Shrinkage and Selection Operator (LASSO). A solution path is the set of all the optimizers with respect to the evolution of the hyperparameter (Lagrange multiplier). The study of the solution path is of great significance in viewing and understanding the profile of the tradeoff between the approximation and regularization terms. If the solution path of a given problem is known, it can help us to find the optimal hyperparameter under a given criterion such as the Akaike Information Criterion. In this paper we present a sufficient condition on ℓ-1 norm penalized least-square problem. Under this sufficient condition, the number of nonzero entries in the optimizer or solution vector increases monotonically when the hyperparameter decreases. We also generalize the result to the often used total variation case, where the ℓ-1 norm is taken over the first order derivative of the solution vector. We prove that the proposed condition has intrinsic connections with the condition given by Donoho, et al and the positive cone condition by Efron el al . However, the proposed condition does not need to assume the sparsity level of the signal as required by Donoho et al's condition, and is easier to verify than Efron, et al's positive cone condition when being used for practical applications.

Robust nonnegative mixed-norm algorithm with weighted <tex>$l_{1}$</tex>-norm regularization

Robust Nonnegative Mixed-Norm Algorithm with Weighted $L_{1}$-Norm Regularization

Online system identification under non-negativity and ℓ1-norm constraints algorithm and weight behavior analysis

Reweighted Nonnegative Least-Mean-Square Algorithm

Improved Nonnegative Adaptive Filtering Algorithms

Variants of Non-Negative Least-Mean-Square Algorithm and Convergence Analysis

Nonnegative Least Mean Mixed-Norm Algorithm: Analysis and Performance.

Nonnegative Least-Mean-Square Algorithm

Kernel Least-Mean Mixed-Norm Algorithm

Steady-State Performance of Non-Negative Least-Mean-Square Algorithm and Its Variants

Block-sparse sign algorithm and its performance analysis

Robust Nonnegative Matrix Factorization via $L_1$ Norm Regularization

Sparse Normalized Subband Adaptive Filter Algorithm with L0-Norm Constraint

Linearized Alternating Directions Method for ℓ1-Norm Inequality Constrained ℓ1-Norm Minimization

A Sufficient Condition on Monotonic Increase of the Number of Nonzero Entry in the Optimizer of L1 Norm Penalized Least-Square Problem

Robust nonnegative matrix factorization via L1 norm regularization by multiplicative updating rules

New Normalized LMS Adaptive Filter with a Variable Regularization Factor

Robust Bias Compensation Method for Sparse Normalized Quasi-Newton Least-Mean with Variable Mixing-Norm Adaptive Filtering

l0 Norm Constraint LMS Algorithm for Sparse System Identification.

$L_{0}$ Norm Constraint LMS Algorithm for Sparse System Identification

Reduced-Space Iteratively Reweighted Second-Order Methods for Nonconvex Sparse Regularization