Abstract:In this paper, we recover sparse signals from their noisy linear measurements by solving nonlinear differential inclusions, which is based on the notion of inverse scale space (ISS) developed in applied mathematics. Our goal here is to bring this idea to address a challenging problem in statistics, \emph{i.e.} finding the oracle estimator which is unbiased and sign-consistent using dynamics. We call our dynamics \emph{Bregman ISS} and \emph{Linearized Bregman ISS}. A well-known shortcoming of LASSO and any convex regularization approaches lies in the bias of estimators. However, we show that under proper conditions, there exists a bias-free and sign-consistent point on the solution paths of such dynamics, which corresponds to a signal that is the unbiased estimate of the true signal and whose entries have the same signs as those of the true signs, \emph{i.e.} the oracle estimator. Therefore, their solution paths are regularization paths better than the LASSO regularization path, since the points on the latter path are biased when sign-consistency is reached. We also show how to efficiently compute their solution paths in both continuous and discretized settings: the full solution paths can be exactly computed piece by piece, and a discretization leads to \emph{Linearized Bregman iteration}, which is a simple iterative thresholding rule and easy to parallelize. Theoretical guarantees such as sign-consistency and minimax optimal $l_2$-error bounds are established in both continuous and discrete settings for specific points on the paths. Early-stopping rules for identifying these points are given. The key treatment relies on the development of differential inequalities for differential inclusions and their discretizations, which extends the previous results and leads to exponentially fast recovering of sparse signals before selecting wrong ones.

An Approach of Regularization Parameter Estimation for Sparse Signal Recovery

An Adaptive Regularization Method for Sparse Representation

A Stable Multi-parameter Thresholding Method for Ill-conditioned Sparse Signal Restoration and Optimal Parameter Choices.

A New Smoothed L0 Regularization Approach for Sparse Signal Recovery

Robust Sparse Recovery via Weakly Convex Regularization

Nonconvex Penalized Regularization for Robust Sparse Recovery in the Presence of $s\alpha S$ Noise

Weakly Convex Regularized Robust Sparse Recovery Methods with Theoretical Guarantees

Anti-Noise-Folding Regularized Subspace Pursuit Recovery Algorithm for Noisy Sparse Signals

Robust Sparse Recovery in Impulsive Noise Via M-Estimator and Non-Convex Regularization

Sorted L1/L2 Minimization for Sparse Signal Recovery

Noisy Sparse Recovery Based on Parameterized Quadratic Programming by Thresholding

Smoothed $$\ell _1$$ℓ1-Regularization-based Line Search for Sparse Signal Recovery

Selection of the Regularization Parameter in the P-Lasso for the Noisy Covariance Model

Noisy Sparsity Recovery Via Differential Equations

New Regularization Method and Iteratively Reweighted Algorithm for Sparse Vector Recovery

A Reweighted Symmetric Smoothed Function Approximating L0-Norm Regularized Sparse Reconstruction Method

A Parameter-Free Estimation Method Based on Low-Rank and Sparse Hybrid Constraints for Scanning Radar Forward-Looking Imaging

Convergence of ℓ2/3 Regularization for Sparse Signal Recovery

Sparse Parameter Identification for Stochastic Systems Based on [math] Regularization

Sorted Minimization for Sparse Signal Recovery

Efficient Recovery of Group-Sparse Signals with Truncated and Reweighted L2, 1-Regularization