Abstract:Recent work by Rauhut and Ward developed a notion of weighted sparsity and a corresponding notion of Restricted Isometry Property for the space of weighted sparse signals. Using these notions, we pose a best weighted sparse approximation problem, i.e. we seek structured sparse solutions to underdetermined systems of linear equations. Many computationally efficient greedy algorithms have been developed to solve the problem of best $s$-sparse approximation. The design of all of these algorithms employ a similar template of exploiting the RIP and computing projections onto the space of sparse vectors. We present an extension of the Iterative Hard Thresholding (IHT) algorithm to solve the weighted sparse approximation problem. This IHT extension employs a weighted analogue of the template employed by all greedy sparse approximation algorithms. Theoretical guarantees are presented and much of the original analysis remains unchanged and extends quite naturally. However, not all the theoretical analysis extends. To this end, we identify and discuss the barrier to extension. Much like IHT, our IHT extension requires computing a projection onto a non-convex space. However unlike IHT and other greedy methods which deal with the classical notion of sparsity, no simple method is known for computing projections onto these weighted sparse spaces. Therefore we employ a surrogate for the projection and analyze its empirical performance.

Ventilation-perfusion relationships in children with heart disease and diminished pulmonary blood flow.

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Half-Space Proximal Stochastic Gradient Method for Group-Sparsity Regularized Problem

Non-Convex Weighted Lp Minimization based Group Sparse Representation Framework for Image Denoising

Subspace Newton method for sparse group optimization problem

Nonconvex Weighted $\ell _p$ Minimization Based Group Sparse Representation Framework for Image Denoising

Analyzing the Group Sparsity Based on the Rank Minimization Methods.

Iterative Hard Thresholding for Weighted Sparse Approximation

Exclusive Sparsity Norm Minimization with Random Groups Via Cone Projection

A minimax optimal approach to high-dimensional double sparse linear regression

Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization.

Whole mitochondrial genome analysis in South Indian patients with Leber's hereditary optic neuropathy.

Image Compressive Sensing Recovery Using Group Sparse Coding Via Non-convex Weighted Lp Minimization.

Smoothing composite proximal gradient algorithm for sparse group Lasso problems with nonsmooth loss functions

Scaled Proximal Gradient Methods for Sparse Optimization Problems

Constrained Mix Sparse Optimization via Hard Thresholding Pursuit

GIST: General Iterative Shrinkage and Thresholding for Non-convex Sparse Learning

OptG: Optimizing Gradient-driven Criteria in Network Sparsity

Matrix Optimization Problem Involving Group Sparsity and Nonnegativity Constraints

A first-order optimization algorithm for statistical learning with hierarchical sparsity structure

Stochastic Iterative Hard Thresholding for Graph-structured Sparsity Optimization