Abstract:The special importance of L 1/2 regularization has been recognized in recent studies on sparse modeling (particularly on compressed sensing). The L 1/2 regularization, however, leads to a nonconvex, nonsmooth, and non-Lipschitz optimization problem that is difficult to solve fast and efficiently. In this paper, through developing a threshoding representation theory for L 1/2 regularization, we propose an iterative half thresholding algorithm for fast solution of L 1/2 regularization, corresponding to the well-known iterative soft thresholding algorithm for L 1 regularization, and the iterative hard thresholding algorithm for L 0 regularization. We prove the existence of the resolvent of gradient of ||x|| 1/2 1/2 , calculate its analytic expression, and establish an alternative feature theorem on solutions of L 1/2 regularization, based on which a thresholding representation of solutions of L 1/2 regularization is derived and an optimal regularization parameter setting rule is formulated. The developed theory provides a successful practice of extension of the well- known Moreau's proximity forward-backward splitting theory to the L 1/2 regularization case. We verify the convergence of the iterative half thresholding algorithm and provide a series of experiments to assess performance of the algorithm. The experiments show that the half algorithm is effective, efficient, and can be accepted as a fast solver for L 1/2 regularization. With the new algorithm, we conduct a phase diagram study to further demonstrate the superiority of L 1/2 regularization over L 1 regularization.

A Constrained Optimization Reformulation and a Feasible Descent Direction Method for $$L_{1/2}$$ L 1 / 2 Regularization

A constrained optimization reformulation and a feasible descent direction method for $$L_{1/2}$$ regularization

A constrained optimization reformulation and a feasible descent direction method for L_1/2 regularization

A constrained optimization reformulation and a feasible descent direction method for $$L_{1/2}$$L1/2 regularization

&Lt;formula Formulatype="inline"> <tex Notation="TeX">$L_{1/2}$</tex></formula> Regularization: A Thresholding Representation Theory and a Fast Solver

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Linearized Alternating Directions Method for ℓ1-Norm Inequality Constrained ℓ1-Norm Minimization

Linearized Alternating Direction Method of Multipliers for Constrained Nonconvex Regularized Optimization.

An Aggressive Reduction on the Complexity of Optimization for Non-Strongly Convex Objectives

The Regularization Continuation Method with an Adaptive Time Step Control for Linearly Constrained Optimization Problems

Avoiding strict saddle points of nonconvex regularized problems

A Constrained Optimization Approach for Lcp

Hard Thresholding Pursuit with Continuation for ℓ0‐regularized Minimizations

Nonstationary Asymptotical Regularization Method with Convex Constraints for Linear Ill-Posed Problems

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs

Solving the L1 regularized least square problem via a box-constrained smooth minimization

A Preconditioned Conjugate Gradient Method with Active Set Strategy for ℓ _1 -Regularized Least Squares

$L_{1/2}$ Regularization: Convergence of Iterative Half Thresholding Algorithm

Lagrangian Regularization Approach to Constrained Optimization Problems

A GRADIENT BASED METHOD FOR THE L-2-L-1/2 MINIMIZATION AND APPLICATION TO COMPRESSIVE SENSING

Penalty Decomposition Methods for $L0$-Norm Minimization