Abstract:In this paper, we recover sparse signals from their noisy linear measurements by solving nonlinear differential inclusions, which is based on the notion of inverse scale space (ISS) developed in applied mathematics. Our goal here is to bring this idea to address a challenging problem in statistics, \emph{i.e.} finding the oracle estimator which is unbiased and sign-consistent using dynamics. We call our dynamics \emph{Bregman ISS} and \emph{Linearized Bregman ISS}. A well-known shortcoming of LASSO and any convex regularization approaches lies in the bias of estimators. However, we show that under proper conditions, there exists a bias-free and sign-consistent point on the solution paths of such dynamics, which corresponds to a signal that is the unbiased estimate of the true signal and whose entries have the same signs as those of the true signs, \emph{i.e.} the oracle estimator. Therefore, their solution paths are regularization paths better than the LASSO regularization path, since the points on the latter path are biased when sign-consistency is reached. We also show how to efficiently compute their solution paths in both continuous and discretized settings: the full solution paths can be exactly computed piece by piece, and a discretization leads to \emph{Linearized Bregman iteration}, which is a simple iterative thresholding rule and easy to parallelize. Theoretical guarantees such as sign-consistency and minimax optimal $l_2$-error bounds are established in both continuous and discrete settings for specific points on the paths. Early-stopping rules for identifying these points are given. The key treatment relies on the development of differential inequalities for differential inclusions and their discretizations, which extends the previous results and leads to exponentially fast recovering of sparse signals before selecting wrong ones.

Dynamical Sparse Signal Recovery with Fixed-Time Convergence

Sparse Bayesian Learning Approach for Discrete Signal Reconstruction

Continuous-Time Sparse Signal Recovery

Convergence Speed of a Dynamical System for Sparse Recovery

Time-varying neurodynamic optimization approaches with fixed-time convergence for sparse signal reconstruction

Sparse Signal Recovery Based on Simulated Annealing

Large Sparse Signal Recovery by Conjugate Gradient Algorithm Based on Smoothing Technique

Dynamic Zero-Point Attracting Projection for Time-Varying Sparse Signal Recovery

Sparse Recovery Via Differential Inclusions

Distributed Continuous and Discrete Time Projection Neurodynamic Approaches for Sparse Recovery.

Noisy Sparsity Recovery Via Differential Equations

Second order self-adaptive dynamical system for sparse signal reconstruction and applications to image recovery

Distributed Computation for Sparse Recovery Via Continuous-Time Neurodynamic Approach

On A Class of Greedy Sparse Recovery Algorithms -- A High Dimensional Approach

Sparsely Corrupted Stimulated Scattering Signals Recovery by Iterative Reweighted Continuous Basis Pursuit

Fast Sparse Recovery Via Non-Convex Optimization

A Unified Primal Dual Active Set Algorithm for Nonconvex Sparse Recovery

Sparse Signal Recovery Via Alternating Projection Method

A Non-convex Approach for Sparse Recovery with Convergence Guarantee

Convergence Of L(2/3) Regularization For Sparse Signal Recovery

Convergence of ℓ2/3 Regularization for Sparse Signal Recovery