Abstract:The ℓ _1 norm is the tight convex relaxation for the ℓ _0 norm and has been successfully applied for recovering sparse signals. However, for problems with fewer samples than required for accurate ℓ _1 recovery, one needs to apply nonconvex penalties such as ℓ _p norm. As one method for solving ℓ _p minimization problems, iteratively reweighted ℓ _1 minimization updates the weight for each component based on the value of the same component at the previous iteration. It assigns large weights on small components in magnitude and small weights on large components in magnitude. The set of the weights is not fixed, and it makes the analysis of this method difficult. In this paper, we consider a weighted ℓ _1 penalty with the set of the weights fixed, and the weights are assigned based on the sort of all the components in magnitude. The smallest weight is assigned to the largest component in magnitude. This new penalty is called nonconvex sorted ℓ _1 . Then we propose two methods for solving nonconvex sorted ℓ _1 minimization problems: iteratively reweighted ℓ _1 minimization and iterative sorted thresholding, and prove that both methods will converge to a local minimizer of the nonconvex sorted ℓ _1 minimization problems. We also show that both methods are generalizations of iterative support detection and iterative hard thresholding, respectively. The numerical experiments demonstrate the better performance of assigning weights by sort compared to assigning by value.

Global Search and Analysis for the Nonconvex Two-Level ℓ₁ Penalty

The convergence of conjugate gradient method with nonmonotone line search

Sparse Estimation Via Lower-Order Penalty Optimization Methods in High-Dimensional Linear Regression.

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Two-level ℓ1 Minimization for Compressed Sensing

Sparse Estimation Via ℓ_q Optimization Method in High-Dimensional Linear Regression

Improving Sparsity and Scalability in Regularized Nonconvex Truncated-Loss Learning Problems

Optimal computational and statistical rates of convergence for sparse nonconvex learning problems

A Non-convex Approach for Sparse Recovery with Convergence Guarantee

GIST: General Iterative Shrinkage and Thresholding for Non-convex Sparse Learning

Fast Learning with Nonconvex L1-2 Regularization.

Nonconvex Sorted ℓ _1 Minimization for Sparse Approximation

On optimal solutions of the constrained $\ell_0$ regularization and its penalty problem

Feature Screening Strategy for Non-Convex Sparse Logistic Regression with Log Sum Penalty

Robust Sparse Recovery Via Non-Convex Optimization

A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems

Structured Sparsity Optimization with Non-Convex Surrogates of L2,0-Norm: A Unified Algorithmic Framework

Subspace Newton method for the $\ell_0$-regularized optimization

Fast Subspace Optimization Method for Nonlinear Inverse Problems in Banach Spaces with Uniformly Convex Penalty Terms

Local and Global Optimality of LP Minimization for Sparse Recovery.

Non-convex Penalized Estimation in High-Dimensional Models with Single-Index Structure