Abstract:The ℓ _1 norm is the tight convex relaxation for the ℓ _0 norm and has been successfully applied for recovering sparse signals. However, for problems with fewer samples than required for accurate ℓ _1 recovery, one needs to apply nonconvex penalties such as ℓ _p norm. As one method for solving ℓ _p minimization problems, iteratively reweighted ℓ _1 minimization updates the weight for each component based on the value of the same component at the previous iteration. It assigns large weights on small components in magnitude and small weights on large components in magnitude. The set of the weights is not fixed, and it makes the analysis of this method difficult. In this paper, we consider a weighted ℓ _1 penalty with the set of the weights fixed, and the weights are assigned based on the sort of all the components in magnitude. The smallest weight is assigned to the largest component in magnitude. This new penalty is called nonconvex sorted ℓ _1 . Then we propose two methods for solving nonconvex sorted ℓ _1 minimization problems: iteratively reweighted ℓ _1 minimization and iterative sorted thresholding, and prove that both methods will converge to a local minimizer of the nonconvex sorted ℓ _1 minimization problems. We also show that both methods are generalizations of iterative support detection and iterative hard thresholding, respectively. The numerical experiments demonstrate the better performance of assigning weights by sort compared to assigning by value.

Weighted Thresholding Homotopy Method for Sparsity Constrained Optimization

Ventilation-perfusion relationships in children with heart disease and diminished pulmonary blood flow.

Adaptive Iterative Hard Thresholding for Least Absolute Deviation Problems with Sparsity Constraints

Scaled Proximal Gradient Methods for Sparse Optimization Problems

Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization.

Dual Iterative Hard Thresholding: from Non-convex Sparse Minimization to Non-smooth Concave Maximization.

A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem

A New Linear Convergence Result for the Iterative Soft Thresholding Algorithm

Minimization Over the Nonconvex Sparsity Constraint Using A Hybrid First-order method

Iterative Hard Thresholding for Weighted Sparse Approximation

Stochastic Iterative Hard Thresholding for Graph-structured Sparsity Optimization

Reduced-Space Iteratively Reweighted Second-Order Methods for Nonconvex Sparse Regularization

Constrained Mix Sparse Optimization via Hard Thresholding Pursuit

Separable Bregman Framework for Sparsity Constrained Nonlinear Optimization

Heavy-ball-based optimal thresholding algorithms for sparse linear inverse problems

A Stable Multi-parameter Thresholding Method for Ill-conditioned Sparse Signal Restoration and Optimal Parameter Choices.

Nonconvex Sorted ℓ _1 Minimization for Sparse Approximation

Optimal $k$-thresholding algorithms for sparse optimization problems

An Iterative Threshold Algorithm of Log-sum Regularization for Sparse Problem

Whole mitochondrial genome analysis in South Indian patients with Leber's hereditary optic neuropathy.

A first-order optimization algorithm for statistical learning with hierarchical sparsity structure