Abstract:In the area of sparse recovery, numerous researches hint that non-convex penalties might induce better sparsity than convex ones, but up until now the non-convex algorithms lack convergence guarantee from the initial solution to the global optimum. This paper aims to provide theoretical guarantee for sparse recovery via non-convex optimization. The concept of weak convexity is incorporated into a class of sparsity-inducing penalties to characterize their non-convexity. It is shown that in a neighborhood of the sparse signal (with radius in inverse proportion to the non-convexity), any local optimum can be regarded as a stable solution. It is further proved that if the non-convexity of the penalty function is below a threshold, the initial solution also belongs to this neighborhood. In addition, The idea of projected (sub)gradient method is generalized to solve this non-convex optimization problem. A uniform approximate projection can also be applied in the projection step to make the algorithm computationally tractable for large scale problems. The theoretical convergence analysis of these methods is provided in the noisy scenario. The result reveals that if the non-convexity is below a threshold, these methods would converge from the initial solution, and the recovered solution is with recovery error linear in both the noise term and the step size. Numerical simulations are performed to test the performance of the proposed approach and verify the theoretical analysis.

A Nonconvex Approach for Structured Sparse Learning.

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Structured Sparsity Optimization with Non-Convex Surrogates of $\ell _{2,0}$ℓ2,0-Norm: A Unified Algorithmic Framework

Structured Sparsity Optimization with Non-Convex Surrogates of L2,0-Norm: A Unified Algorithmic Framework

Sparse Estimation Via ℓ_q Optimization Method in High-Dimensional Linear Regression

A Non-convex Approach for Sparse Recovery with Convergence Guarantee

Low-rank structure learning via nonconvex heuristic recovery.

A Systematic Review of Structured Sparse Learning

Sparse Estimation Via Lower-Order Penalty Optimization Methods in High-Dimensional Linear Regression.

Analysis non-sparse recovery for non-convex relaxed ℓ q minimization

GIST: General Iterative Shrinkage and Thresholding for Non-convex Sparse Learning

Sparse Learning for Large-Scale and High-Dimensional Data: A Randomized Convex-Concave Optimization Approach

Robust Sparse Recovery Via Non-Convex Optimization

From Least Squares to Sparse: A Non-Convex Approach with Guarantee

The Convergence Guarantees of a Non-Convex Approach for Sparse Recovery.

Convex Relaxation Algorithm for a Structured Simultaneous Low-Rank and Sparse Recovery Problem

An Alternating Proximal Splitting Method With Global Convergence For Nonconvex Structured Sparsity Optimization

A Greedy Multistage Convex Relaxation Algorithm Applied to Structured Group Sparse Reconstruction Problems Based on Iterative Support Detection

A multi-stage convex relaxation approach to noisy structured low-rank matrix recovery

Robust Sparse Recovery Via Weakly Convex Optimization in Impulsive Noise