Abstract:We provide a geometric approach to the lasso. We study the tangency of the level sets of the least square objective function with the polyhedral boundary sets $B(t)$ of the parameters in $\mathbb R^p$ with the $\ell_1$ norm equal to $t$. Here $t$ decreases from the value $\hat t$, which corresponds to the actual, nonconstrained minimizer of the least square objective function, denoted by $\hat\beta$. We derive closed exact formulae for the solution of the lasso under the full rank assumption. Our method does not assume iterative numerical procedures and it is, thus, computationally more efficient than the existing algorithms for solving the lasso. We also establish several important general properties of the solutions of the lasso. We prove that each lasso solution form a simple polygonal chain in $\mathbb{R}^p$ with $\hat\beta$ and the origin as the endpoints. There are no two segments of the polygonal chain that are parallel. We prove that such a polygonal chain can intersect interiors of more than one orthant in $\mathbb{R}^p$, but it cannot intersect interiors of more than $p$ orthants, which is, in general, the best possible estimate for non-normalized data. We prove that if a polygonal chain passes from the interior of one to the interior of another orthant, then it never again returns to the interior of the former. The intersection of a chain and the interior of an orthant coincides with a segment minus its end points, which belongs to a ray having $\hat\beta$ as its initial point. We illustrate the results using real data examples as well as especially crafted examples with hypothetical data. Already in $p=2$ case we show a striking difference in the maximal number of quadrants a polygonal chain of a lasso solution can intersect in the case of normalized data, which is $1$ vs. nonnormalized data, which is $2$.

On LARS/Homotopy Equivalence Conditions for Over-Determined LASSO.

On Model Selection Consistency of Lasso for High-Dimensional Ising Models

Average case analysis of Lasso under ultra-sparse conditions

Adaptive Lasso in high-dimensional settings

A Sufficient Condition on Monotonic Increase of the Number of Nonzero Entry in the Optimizer of L1 Norm Penalized Least-Square Problem

Homotopy Based Algorithms for $\ell _{\scriptscriptstyle 0}$ -Regularized Least-Squares

The Adaptive Lasso and Its Oracle Properties

A Greedy Homotopy Method for Regression with Nonconvex Constraints

A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem

New Error Analysis for Lasso

Nonnegative-lasso and application in index tracking

Algebraic and Geometric Properties of $\mathcal{L}^n_+$-Semipositive Matrices and $\mathcal{L}^n_+$-Semipositive Cones

A local stochastic Lipschitz condition with application to Lasso for high dimensional generalized linear models

Study on L_1 over L_2 Minimization for Nonnegative Signal Recovery

Study on <i>L</i><sub>1</sub> over <i>L</i><sub>2</sub> Minimization for Nonnegative Signal Recovery

Optimality Conditions for Problems over Symmetric Cones and a Simple Augmented Lagrangian Method

Non-asymptotic Oracle Inequalities for the Lasso and Group Lasso in high dimensional logistic model

Fast Exact Conformalization of Lasso using Piecewise Linear Homotopy

High-dimensional generalized linear models and the lasso

Convergence of LCA Flows to (C)LASSO Solutions

Geometric Approach and Closed Exact Formulae for the Lasso