Abstract:We focus on solving the clustered lasso problem, which is a least squares problem with the $ell_1$-type penalties imposed on both the coefficients and their pairwise differences to learn the group structure of the regression parameters. Here we first reformulate the clustered lasso regularizer as a weighted ordered-lasso regularizer, which is essential in reducing the computational cost from $O(n^2)$ to $O(nlog (n))$. We then propose an inexact semismooth Newton augmented Lagrangian (SSNAL) algorithm to solve the clustered lasso problem or its dual via this equivalent formulation, depending on whether the sample size is larger than the dimension of the features. An essential component of the SSNAL algorithm is the computation of the generalized Jacobian of the proximal mapping of the clustered lasso regularizer. Based on the new formulation, we derive an efficient procedure for its computation. Comprehensive results on the global convergence and local linear convergence of the SSNAL algorithm are established. For the purpose of exposition and comparison, we also summarize/design several first-order methods that can be used to solve the problem under consideration, but with the key improvement from the new formulation of the clustered lasso regularizer. As a demonstration of the applicability of our algorithms, numerical experiments on the clustered lasso problem are performed. The experiments show that the SSNAL algorithm substantially outperforms the best alternative algorithm for the clustered lasso problem.

A Greedy Homotopy Method for Regression with Nonconvex Constraints

Adaptive Lasso in high-dimensional settings

A Fast Solution to the Lasso Problem with Equality Constraints

A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem

Robust Fused Lasso Penalized Huber Regression with Nonasymptotic Property and Implementation Studies

Fast Exact Conformalization of Lasso using Piecewise Linear Homotopy

Generalized Newton–Raphson Algorithm for High Dimensional LASSO Regression

Lasso Regression: Estimation and Shrinkage via Limit of Gibbs Sampling

Lasso with Latents: Efficient Estimation, Covariate Rescaling, and Computational-Statistical Gaps

A Tuning-free Robust and Efficient Approach to High-dimensional Regression

A Sparse-Group Lasso

A Fast Method for Lasso and Logistic Lasso

The Trimmed Lasso: Sparsity and Robustness

Efficient Inference on High-Dimensional Linear Models with Missing Outcomes

Proximal Iteration for Nonlinear Adaptive Lasso

A Fast Coordinate Descent Method for High-Dimensional Non-Negative Least Squares using a Unified Sparse Regression Framework

A first-order optimization algorithm for statistical learning with hierarchical sparsity structure

Scaled minimax optimality in high-dimensional linear regression: A non-convex algorithmic regularization approach

Efficient sparse Hessian based algorithms for the clustered lasso problem.

A Constructive Approach to High-dimensional Regression