Fast Learning with Nonconvex L1-2 Regularization.

Quanming Yao,James T. Kwok,Xiawei Guo
DOI: https://doi.org/10.48550/arxiv.1610.09461
2016-01-01
Abstract:Convex regularizers are often used for sparse learning. They are easy to optimize, but can lead to inferior prediction performance. The difference of ℓ_1 and ℓ_2 (ℓ_1-2) regularizer has been recently proposed as a nonconvex regularizer. It yields better recovery than both ℓ_0 and ℓ_1 regularizers on compressed sensing. However, how to efficiently optimize its learning problem is still challenging. The main difficulty is that both the ℓ_1 and ℓ_2 norms in ℓ_1-2 are not differentiable, and existing optimization algorithms cannot be applied. In this paper, we show that a closed-form solution can be derived for the proximal step associated with this regularizer. We further extend the result for low-rank matrix learning and the total variation model. Experiments on both synthetic and real data sets show that the resultant accelerated proximal gradient algorithm is more efficient than other noncovex optimization algorithms.
What problem does this paper attempt to address?