Abstract:We study the minimization problem of a non-convex sparsity promoting penalty function, the transformed $l_1$ (TL1), and its application in compressed sensing (CS). The TL1 penalty interpolates $l_0$ and $l_1$ norms through a nonnegative parameter $a \in (0,+\infty)$, similar to $l_p$ with $p \in (0,1]$, and is known to satisfy unbiasedness, sparsity and Lipschitz continuity properties. We first consider the constrained minimization problem and discuss the exact recovery of $l_0$ norm minimal solution based on the null space property (NSP). We then prove the stable recovery of $l_0$ norm minimal solution if the sensing matrix $A$ satisfies a restricted isometry property (RIP). Next, we present difference of convex algorithms for TL1 (DCATL1) in computing TL1-regularized constrained and unconstrained problems in CS. The inner loop concerns an $l_1$ minimization problem on which we employ the Alternating Direction Method of Multipliers (ADMM). For the unconstrained problem, we prove convergence of DCATL1 to a stationary point satisfying the first order optimality condition. In numerical experiments, we identify the optimal value $a=1$, and compare DCATL1 with other CS algorithms on two classes of sensing matrices: Gaussian random matrices and over-sampled discrete cosine transform matrices (DCT). We find that for both classes of sensing matrices, the performance of DCATL1 algorithm (initiated with $l_1$ minimization) always ranks near the top (if not the top), and is the most robust choice insensitive to the conditioning of the sensing matrix $A$. DCATL1 is also competitive in comparison with DCA on other non-convex penalty functions commonly used in statistics with two hyperparameters.

Tikhonov Regularization with L^0-Term Complementing a Convex Penalty: L^1 Convergence under Sparsity Constraints

Multi-parameter Tikhonov Regularization with the ℓ 0 Sparsity Constraint

Multi-parameter Tikhonov regularization with the ` sparsity constraint: ` convergence

An iteration regularization method with general convex penalty for nonlinear inverse problems in Banach spaces

Characterizing the SLOPE Trade-off: A Variational Perspective and the Donoho-Tanner Limit

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Tikhonov regularized exterior penalty dynamics for constrained variational inequalities

High-dimensional Inference Via Lipschitz Sparsity-Yielding Regularizers.

Beyond convergence rates: Exact recovery with Tikhonov regularization with sparsity constraints

Sparse Reconstruction by Means of the Standard Tikhonov Regularization

An Analysis of Tikhonov Regularization for Nonlinear Ill-Posed Problems under a General Smoothness Assumption

Penalty Decomposition Methods for $L0$-Norm Minimization

Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces

On the Generalized Discrepancy Principle for Tikhonov Regularization in Hilbert Scales

Condition Numbers and Perturbation Analysis for the Tikhonov Regularization of Discrete Ill-Posed Problems.

Convergence Rate Analysis of Tikhonov Regularization for Nonlinear Ill-Posed Problems with Noisy Operators

Concentration Estimates for Learning with ℓ1-Regularizer and Data Dependent Hypothesis Spaces

Nonconvex Penalization Using Laplace Exponents and Concave Conjugates.

Low-Rank Tensor Completion via Novel Sparsity-Inducing Regularizers

Minimization of Transformed $L_1$ Penalty: Theory, Difference of Convex Function Algorithm, and Robust Application in Compressed Sensing

Two-level ℓ1 Minimization for Compressed Sensing