Abstract:We focus on solving the clustered lasso problem, which is a least squares problem with the $ell_1$-type penalties imposed on both the coefficients and their pairwise differences to learn the group structure of the regression parameters. Here we first reformulate the clustered lasso regularizer as a weighted ordered-lasso regularizer, which is essential in reducing the computational cost from $O(n^2)$ to $O(nlog (n))$. We then propose an inexact semismooth Newton augmented Lagrangian (SSNAL) algorithm to solve the clustered lasso problem or its dual via this equivalent formulation, depending on whether the sample size is larger than the dimension of the features. An essential component of the SSNAL algorithm is the computation of the generalized Jacobian of the proximal mapping of the clustered lasso regularizer. Based on the new formulation, we derive an efficient procedure for its computation. Comprehensive results on the global convergence and local linear convergence of the SSNAL algorithm are established. For the purpose of exposition and comparison, we also summarize/design several first-order methods that can be used to solve the problem under consideration, but with the key improvement from the new formulation of the clustered lasso regularizer. As a demonstration of the applicability of our algorithms, numerical experiments on the clustered lasso problem are performed. The experiments show that the SSNAL algorithm substantially outperforms the best alternative algorithm for the clustered lasso problem.

The Linearized Alternating Direction Method of Multipliers for Sparse Group LAD Model.

Average case analysis of Lasso under ultra-sparse conditions

Adaptive Lasso in high-dimensional settings

A Sparse-Group Lasso

A New Linearized Alternating Direction Method of Multipliers with Adaptive Step Size and Its Inexact Version for Fused Lasso Regression Model

Robust Regression Shrinkage and Consistent Variable Selection Through the LAD-Lasso

Linearized Alternating Direction Method of Multipliers for Constrained Nonconvex Regularized Optimization.

A new penalized least absolute deviation model for high dimensional sparse linear regression and an efficient sequential linear programming algorithm

Fused LASSO Penalized Least Absolute Deviation Estimator for High Dimensional Linear Regression

Group Sparse Optimization By Alternating Direction Method

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

Multi-block linearized alternating direction method for sparse fused Lasso modeling problems

A constructive approach to L

Penalized and Constrained LAD Estimation in Fixed and High Dimension

Efficient sparse Hessian based algorithms for the clustered lasso problem.

The linearized alternating direction method of multipliers for low-rank and fused LASSO matrix regression model

Efficient and Accelerated Online Learning for Sparse Group Lasso

Convergence and Sparsity of Lasso and Group Lasso in High-Dimensional Generalized Linear Models

Method for Solving LASSO Problem Based on Multidimensional Weight.

A Constructive Approach to L0 Penalized Regression