Abstract:Group Lasso is an important sparse regression method in machine learning which encourages selecting key explanatory factors in a grouped manner because of the use of L-2,1 norm. In real-world learning tasks, some chunks of data would be added into or removed from the training set in sequence due to the existence of new or obsolete historical data, which is normally called dynamic or lifelong learning scenario. However, most of existing algorithms of group Lasso are limited to offline updating, and only one is online algorithm which can only handle newly added samples inexactly. Due to the complexity of L-2,1 norm, how to achieve accurate chunk incremental and decremental learning efficiently for group Lasso is still an open question. To address this challenging problem, in this paper, we propose a novel accurate dynamic updating algorithm for group Lasso by utilizing the technique of Ordinary Differential Equations (ODEs), which can incorporate or eliminate a chunk of samples from original training set without retraining the model from scratch. Specifically, we introduce a new formulation to reparameterize the adjustment procedures of chunk incremental and decremental learning simultaneously. Based on the new formulation, we propose a path following algorithm for group Lasso regarding to the adjustment parameter. Importantly, we prove that our path following algorithm can exactly track the piecewise smooth solutions thanks to the technique of ODEs, so that the accurate chunk incremental and decremental learning can be achieved. Extensive experimental results not only confirm the effectiveness of proposed algorithm for the chunk incremental and decremental learning, but also validate its efficiency compared to the existing offline and online algorithms.

Efficient and Accelerated Online Learning for Sparse Group Lasso

Online Kernel Learning with a Near Optimal Sparsity Bound

Group Lasso Online Learning

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

A Communication-Efficient Parallel Method for Group-Lasso.

The Linearized Alternating Direction Method of Multipliers for Sparse Group LAD Model.

Efficient Online Learning for Large-Scale Sparse Kernel Logistic Regression

A Fast and Scalable Pathwise-Solver for Group Lasso and Elastic Net Penalized Regression via Block-Coordinate Descent

A Sparse-Group Lasso

Dual Averaging Method For Online Graph-Structured Sparsity

Efficient sparse Hessian based algorithms for the clustered lasso problem.

Asynchronous Doubly Stochastic Group Regularized Learning

Chunk Dynamic Updating for Group Lasso with ODEs.

A Novel Differentially Private Online Learning Algorithm for Group Lasso in Big Data

A Dual Perspective of Sparse and Robust Online Learning Algorithm

Efficient Sparse Semismooth Newton Methods for the Clustered Lasso Problem

A PROXIMAL POINT DUAL NEWTON ALGORITHM FOR SOLVING GROUP GRAPHICAL LASSO PROBLEMS

Accelerated Online Learning for Collaborative Filtering and Recommender Systems

Online Group Feature Selection

Adaptive Optimizers with Sparse Group Lasso for Neural Networks in CTR Prediction