Abstract:We focus on solving the clustered lasso problem, which is a least squares problem with the $\ell_1$-type penalties imposed on both the coefficients and their pairwise differences to learn the group structure of the regression parameters. Here we first reformulate the clustered lasso regularizer as a weighted ordered-lasso regularizer, which is essential in reducing the computational cost from $O(n^2)$ to $O(n\log (n))$. We then propose an inexact semismooth Newton augmented Lagrangian ({\sc Ssnal}) algorithm to solve the clustered lasso problem or its dual via this equivalent formulation, depending on whether the sample size is larger than the dimension of the features. An essential component of the {\sc Ssnal} algorithm is the computation of the generalized Jacobian of the proximal mapping of the clustered lasso regularizer. Based on the new formulation, we derive an efficient procedure for its computation. Comprehensive results on the global convergence and local linear convergence of the {\sc Ssnal} algorithm are established. For the purpose of exposition and comparison, we also summarize/design several first-order methods that can be used to solve the problem under consideration, but with the key improvement from the new formulation of the clustered lasso regularizer. As a demonstration of the applicability of our algorithms, numerical experiments on the clustered lasso problem are performed. The experiments show that the {\sc Ssnal} algorithm substantially outperforms the best alternative algorithm for the clustered lasso problem.

A Proximal Point Algorithm For Log-Determinant Optimization With Group Lasso Regularization

A PROXIMAL POINT DUAL NEWTON ALGORITHM FOR SOLVING GROUP GRAPHICAL LASSO PROBLEMS

Solving Log-Determinant Optimization Problems by a Newton-CG Primal Proximal Point Algorithm

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

The Linearized Alternating Direction Method of Multipliers for Sparse Group LAD Model.

Efficient sparse Hessian based algorithms for the clustered lasso problem.

Efficient Sparse Semismooth Newton Methods for the Clustered Lasso Problem

Dual Newton Proximal Point Algorithm for Solution Paths of the L1-Regularized Logistic Regression

Smoothing composite proximal gradient algorithm for sparse group Lasso problems with nonsmooth loss functions

A Semismooth Newton-CG Based Dual PPA for Matrix Spectral Norm Approximation Problems

An adaptive proximal point algorithm framework and application to large-scale optimization

Anisotropic Proximal Point Algorithm

Group Sparse Optimization By Alternating Direction Method

Linearized Symmetric Multi-Block ADMM with Indefinite Proximal Regularization and Optimal Proximal Parameter

GLOBAL OPTIMIZATION FOR NON-CONVEX PROGRAMS VIA CONVEX PROXIMAL POINT METHOD

A proximal-gradient method for problems with overlapping group-sparse regularization: support identification complexity

An Alternating Manifold Proximal Gradient Method for Sparse Principal Component Analysis and Sparse Canonical Correlation Analysis

A relaxed customized proximal point algorithm for separable convex programming

General Parameterized Proximal Point Algorithm with Applications in Statistical Learning

Subspace Clustering Using Log-determinant Rank Approximation