Abstract:We focus on solving the clustered lasso problem, which is a least squares problem with the $ell_1$-type penalties imposed on both the coefficients and their pairwise differences to learn the group structure of the regression parameters. Here we first reformulate the clustered lasso regularizer as a weighted ordered-lasso regularizer, which is essential in reducing the computational cost from $O(n^2)$ to $O(nlog (n))$. We then propose an inexact semismooth Newton augmented Lagrangian (SSNAL) algorithm to solve the clustered lasso problem or its dual via this equivalent formulation, depending on whether the sample size is larger than the dimension of the features. An essential component of the SSNAL algorithm is the computation of the generalized Jacobian of the proximal mapping of the clustered lasso regularizer. Based on the new formulation, we derive an efficient procedure for its computation. Comprehensive results on the global convergence and local linear convergence of the SSNAL algorithm are established. For the purpose of exposition and comparison, we also summarize/design several first-order methods that can be used to solve the problem under consideration, but with the key improvement from the new formulation of the clustered lasso regularizer. As a demonstration of the applicability of our algorithms, numerical experiments on the clustered lasso problem are performed. The experiments show that the SSNAL algorithm substantially outperforms the best alternative algorithm for the clustered lasso problem.

Convergence Analysis of Rival Penalized Competitive Learning (RPCL) Algorithm

The Correct Convergence of the Rival Penalized Competitive Learning (rpcl) Algorithm

A Cost-Function Approach to Rival Penalized Competitive Learning (RPCL)

The mahalanobis distance based rival penalized competitive learning algorithm

Convergence Rate of Primal-Dual Approach to Constrained Reinforcement Learning with Softmax Policy

Convergence Analysis of Projected Gradient Descent for Schatten-p Nonconvex Matrix Recovery

Convergence Theorems for a Class of Learning Algorithms with VLRPs

Application of Dynamic Rival Penalized Competitive Learning on the Clustering Analysis of Seismic Data.

Deep Rival Penalized Competitive Learning for Low-resolution Face Recognition

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

A Concave Conjugate Approach for Nonconvex Penalized Regression with the MCP Penalty.

Convergence rates of stochastic gradient method with independent sequences of step-size and momentum weight

On Convergence Rates of Robust Adaptive Game Theoretic Learning Algorithms

Convergence Theorems for the Kohonen Feature Mapping Algorithms with VLRPs

A distributed cooperative learning algorithm based on Zero-Gradient-Sum strategy using Radial Basis Function Network

Coordinate Descent Algorithm for Ramp Loss Linear Programming Support Vector Machines

On the Convergence of Bound Optimization Algorithms

Latent Low-Rank Representation With Weighted Distance Penalty for Clustering

Convergence Analysis of an Adaptively Regularized Natural Gradient Method

Fast and Strong Convergence of Online Learning Algorithms.

Efficient sparse Hessian based algorithms for the clustered lasso problem.