Abstract:In this paper, we recover sparse signals from their noisy linear measurements by solving nonlinear differential inclusions, which is based on the notion of inverse scale space (ISS) developed in applied mathematics. Our goal here is to bring this idea to address a challenging problem in statistics, \emph{i.e.} finding the oracle estimator which is unbiased and sign-consistent using dynamics. We call our dynamics \emph{Bregman ISS} and \emph{Linearized Bregman ISS}. A well-known shortcoming of LASSO and any convex regularization approaches lies in the bias of estimators. However, we show that under proper conditions, there exists a bias-free and sign-consistent point on the solution paths of such dynamics, which corresponds to a signal that is the unbiased estimate of the true signal and whose entries have the same signs as those of the true signs, \emph{i.e.} the oracle estimator. Therefore, their solution paths are regularization paths better than the LASSO regularization path, since the points on the latter path are biased when sign-consistency is reached. We also show how to efficiently compute their solution paths in both continuous and discretized settings: the full solution paths can be exactly computed piece by piece, and a discretization leads to \emph{Linearized Bregman iteration}, which is a simple iterative thresholding rule and easy to parallelize. Theoretical guarantees such as sign-consistency and minimax optimal $l_2$-error bounds are established in both continuous and discrete settings for specific points on the paths. Early-stopping rules for identifying these points are given. The key treatment relies on the development of differential inequalities for differential inclusions and their discretizations, which extends the previous results and leads to exponentially fast recovering of sparse signals before selecting wrong ones.

High-dimensional subset recovery in noise: Sparsified measurements without loss of statistical efficiency

Low Rank Matrix Recovery with Adversarial Sparse Noise

I T ] 8 S ep 2 01 3 Sparse Recovery with Coherent Tight Frames via Analysis Dantzig Selector and Analysis LASSO ∗

Sparse Recovery with Coherent Tight Frame via Analysis Dantzig Selector and Analysis LASSO

Nonuniform Support Recovery from Noisy Random Measurements by Orthogonal Matching Pursuit.

Sparse Signals Recovery from Noisy Measurements by Orthogonal Matching Pursuit

Sharp Support Recovery from Noisy Random Measurements by L1 minimization

Robust Sparse Recovery with Sparse Bernoulli matrices via Expanders

Noisy Sparsity Recovery Via Differential Equations

Sequential Testing for Sparse Recovery

Recovery of Signals with Low Density

Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors

Noisy recovery from random linear observations: Sharp minimax rates under elliptical constraints

Sparse Support Recovery with Non-smooth Loss Functions

Sparse Recovery of Multiple Measurement Vectors in Impulsive Noise: A Smooth Block Successive Minimization Algorithm

Robust Sparse Recovery in Impulsive Noise Via $\ell _p$ -$\Ell _1$ Optimization

Support Recovery of Sparse Signals from a Mixture of Linear Measurements

Robustness of Sparse Recovery Via &Lt;inline-Formula> &Lt;tex-Math Notation="latex">$f$ &Lt;/tex-Math></inline-formula>-minimization: A Topological Viewpoint

Sparse Recovery Via Differential Inclusions

On unified view of nullspace-type conditions for recoveries associated with general sparsity structures

Support Recovery in the Phase Retrieval Model: Information-Theoretic Fundamental Limits