Abstract:This paper introduces new solvers for efficiently computing solutions to large-scale inverse problems with group sparsity regularization, including both non- overlapping and overlapping groups. Group sparsity regularization refers to a type of structured sparsity regularization, where the goal is to impose additional structure in the regularization process by assigning variables to predefined groups that may represent graph or network structures. Special cases of group sparsity regularization include l1 and isotropic total variation regularization. In this work, we develop hybrid projection methods based on flexible Krylov subspaces, where we first recast the group sparsity regularization term as a sequence of 2-norm penalization terms using adaptive regularization matrices in an iterative reweighted norm fashion. Then, we exploit flexible preconditioning techniques to efficiently incorporate the weight updates. The main advantages of these methods are that they are computationally efficient (leveraging the advantages of flexible methods), they are general (and therefore very easily adaptable to new regularization term choices), and they are able to select the regularization parameters automatically and adaptively (exploiting the advantages of hybrid methods). Extensions to multiple regularization terms and solution decomposition frameworks (e.g., for anomaly detection) are described, and a variety of numerical examples demonstrate both the efficiency and accuracy of the proposed approaches compared to existing solvers.

Efficient Inexact Proximal Gradient Algorithms for Structured Sparsity-Inducing Norm.

Faster Training Algorithms for Structured Sparsity-Inducing Norm

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Smoothing Proximal Gradient Method for General Structured Sparse Learning

A proximal-gradient method for problems with overlapping group-sparse regularization: support identification complexity

Smoothing composite proximal gradient algorithm for sparse group Lasso problems with nonsmooth loss functions

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization

Efficient inexact proximal gradient algorithm for nonconvex problems

An Alternating Proximal Splitting Method With Global Convergence For Nonconvex Structured Sparsity Optimization

A first-order optimization algorithm for statistical learning with hierarchical sparsity structure

On Proximal Gradient Method for the Convex Problems Regularized with the Group Reproducing Kernel Norm

Efficient Inexact Proximal Gradient Algorithm for Nonconvex Problems.

Half-Space Proximal Stochastic Gradient Method for Group-Sparsity Regularized Problem

Inexact Accelerated Proximal Gradient Algorithms for Matrix L_{2,1}-Norm Minimization Problem in Multi-Task Feature Learning

Efficient Projected Gradient Methods for L 0 Constrained Optimization

Online Proximal Learning over Jointly Sparse Multitask Networks with $\ell _{\infty, 1}$ Regularization

Flexible Krylov Methods for Group Sparsity Regularization

Computing the Proximal Operator of the $\ell_{1,q}$-norm for Group Sparsity

Proximal Iteration for Nonlinear Adaptive Lasso