GIST: General Iterative Shrinkage and Thresholding for Non-convex Sparse Learning

Pinghua Gong,Changshui Zhang,Zhaosong Lu,Jianhua Huang,Jieping Ye

2013-01-01

Abstract:Learning sparse representations has very important applications in real-world problems. In the last decade, ℓ1-norm based sparse learning [10, 3], by solving a convex optimization problem, has been extensively studied and successfully applied to many areas including signal & image processing [1, 13], computer vision [12], biomedical informatics [9] and so on. However, recent theoretical investigations have shown that ℓ1-norm based sparse learning achieves suboptimal performance in many cases [2, 15, 16]. To this end, many non-convex regularized sparse learning formulations have been proposed and shown their superiority over their convex counterparts in several sparse learning settings. In these non-convex sparse learning formulations, many non-convex regularizers (penalties) are employed, which include ℓq-norm (0< q< 1)[5], Smoothly Clipped Absolute Deviation (SCAD)[4], Log-Sum Penalty (LSP)[2 …