Abstract:This paper introduces new solvers for efficiently computing solutions to large-scale inverse problems with group sparsity regularization, including both non- overlapping and overlapping groups. Group sparsity regularization refers to a type of structured sparsity regularization, where the goal is to impose additional structure in the regularization process by assigning variables to predefined groups that may represent graph or network structures. Special cases of group sparsity regularization include l1 and isotropic total variation regularization. In this work, we develop hybrid projection methods based on flexible Krylov subspaces, where we first recast the group sparsity regularization term as a sequence of 2-norm penalization terms using adaptive regularization matrices in an iterative reweighted norm fashion. Then, we exploit flexible preconditioning techniques to efficiently incorporate the weight updates. The main advantages of these methods are that they are computationally efficient (leveraging the advantages of flexible methods), they are general (and therefore very easily adaptable to new regularization term choices), and they are able to select the regularization parameters automatically and adaptively (exploiting the advantages of hybrid methods). Extensions to multiple regularization terms and solution decomposition frameworks (e.g., for anomaly detection) are described, and a variety of numerical examples demonstrate both the efficiency and accuracy of the proposed approaches compared to existing solvers.

Faster Training Algorithms for Structured Sparsity-Inducing Norm

Efficient Inexact Proximal Gradient Algorithms for Structured Sparsity-Inducing Norm.

Smoothing Proximal Gradient Method for General Structured Sparse Learning

A first-order optimization algorithm for statistical learning with hierarchical sparsity structure

Flexible Krylov Methods for Group Sparsity Regularization

An Alternating Proximal Splitting Method With Global Convergence For Nonconvex Structured Sparsity Optimization

Structured Sparsity Optimization with Non-Convex Surrogates of L2,0-Norm: A Unified Algorithmic Framework

Online Proximal Learning over Jointly Sparse Multitask Networks with $\ell _{\infty, 1}$ Regularization

Structured Sparsity Optimization with Non-Convex Surrogates of $\ell _{2,0}$ℓ2,0-Norm: A Unified Algorithmic Framework

Convergence Analyses on Sparse Feedforward Neural Networks Via Group Lasso Regularization.

A proximal-gradient method for problems with overlapping group-sparse regularization: support identification complexity

Sparse Feature Grouping Based on $\ell_{1/2}$ Norm Regularization

Smoothing composite proximal gradient algorithm for sparse group Lasso problems with nonsmooth loss functions

Group Sparse Optimization By Alternating Direction Method

An Efficient Algorithm for Dictionary Learning with a Mixed-Norm Regularizer for Sparsity Based on Proximal Operator.

Exclusive Feature Learning on Arbitrary Structures via l(1,2)-norm

Fast Learning with Nonconvex L1-2 Regularization.

A sparse-group lasso

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems