Abstract:We provide a geometric approach to the lasso. We study the tangency of the level sets of the least square objective function with the polyhedral boundary sets $B(t)$ of the parameters in $\mathbb R^p$ with the $\ell_1$ norm equal to $t$. Here $t$ decreases from the value $\hat t$, which corresponds to the actual, nonconstrained minimizer of the least square objective function, denoted by $\hat\beta$. We derive closed exact formulae for the solution of the lasso under the full rank assumption. Our method does not assume iterative numerical procedures and it is, thus, computationally more efficient than the existing algorithms for solving the lasso. We also establish several important general properties of the solutions of the lasso. We prove that each lasso solution form a simple polygonal chain in $\mathbb{R}^p$ with $\hat\beta$ and the origin as the endpoints. There are no two segments of the polygonal chain that are parallel. We prove that such a polygonal chain can intersect interiors of more than one orthant in $\mathbb{R}^p$, but it cannot intersect interiors of more than $p$ orthants, which is, in general, the best possible estimate for non-normalized data. We prove that if a polygonal chain passes from the interior of one to the interior of another orthant, then it never again returns to the interior of the former. The intersection of a chain and the interior of an orthant coincides with a segment minus its end points, which belongs to a ray having $\hat\beta$ as its initial point. We illustrate the results using real data examples as well as especially crafted examples with hypothetical data. Already in $p=2$ case we show a striking difference in the maximal number of quadrants a polygonal chain of a lasso solution can intersect in the case of normalized data, which is $1$ vs. nonnormalized data, which is $2$.

Efficient Lasso Training from a Geometrical Perspective.

Geometric Approach and Closed Exact Formulae for the Lasso

New Error Analysis for Lasso

A Fast Method for Lasso and Logistic Lasso

A Fast Solution to the Lasso Problem with Equality Constraints

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

Efficient sparse Hessian based algorithms for the clustered lasso problem.

Fused LASSO Penalized Least Absolute Deviation Estimator for High Dimensional Linear Regression

A Fast Iterative Single Data Approach to Training Unconstrained Least Squares Support Vector Machines

Generalized Newton–Raphson Algorithm for High Dimensional LASSO Regression

Lass-0: sparse non-convex regression by local search

Regularized least weighted squares estimator in linear regression

The Trimmed Lasso: Sparsity and Robustness

A Robust Zero-point Attraction LMS Algorithm on Near Sparse System Identification

Unified LASSO estimation by least squares approximation

A cost-sensitive constrained Lasso

Lasso with Latents: Efficient Estimation, Covariate Rescaling, and Computational-Statistical Gaps

A new penalized least absolute deviation model for high dimensional sparse linear regression and an efficient sequential linear programming algorithm

Fast Exact Conformalization of Lasso using Piecewise Linear Homotopy

A constructive approach to L

A Fast and Scalable Pathwise-Solver for Group Lasso and Elastic Net Penalized Regression via Block-Coordinate Descent