Abstract:Sparse optimization has seen its advances in recent decades. For scenarios where the true sparsity is unknown, regularization turns out to be a promising solution. Two popular non-convex regularizations are the so-called $L_0$ norm and $L_q$ norm with $q\in(0,1)$, giving rise to extensive research on their induced optimization. However, the majority of these work centered around the main function that is twice continuously differentiable and the best convergence rate for an algorithm solving the optimization with $q\in(0,1)$ is superlinear. This paper explores the $L_q$ norm regularized optimization in a unified way for any $q\in[0,1)$, where the main function has a semismooth gradient. In particular, we establish the first-order and the second-order optimality conditions under mild assumptions and then integrate the proximal operator and semismooth Newton method to develop a proximal semismooth Newton pursuit algorithm. Under the second sufficient condition, the whole sequence generated by the algorithm converges to a unique local minimizer. Moreover, the convergence is superlinear and quadratic if the gradient of the main function is semismooth and strongly semismooth at the local minimizer, respectively. Hence, this paper accomplishes the quadratic rate for an algorithm designed to solve the $L_q$ norm regularization problem for any $q\in(0,1)$. Finally, some numerical experiments have showcased its nice performance when compared with several existing solvers.

Online Efficient Learning with Quantized Klms and L-1 Regularization

Online Efficient Learning with Quantized KLMS and L<inf>1</inf> Regularization

Quantised Kernel Least Mean Square with Desired Signal Smoothing

A Quantized Kernel Least Mean Square Scheme with Entropy-Guided Learning for Intelligent Data Analysis

Quantized Kernel Least Mean Square Algorithm

Density-Dependent Quantized Kernel Least Mean Square

Online Kernel Learning with a Near Optimal Sparsity Bound

Quantized Kernel Recursive Least Squares Algorithm.

A Class of Weighted Quantized Kernel Recursive Least Squares Algorithms.

QC-ODKLA: Quantized and Communication-Censored Online Decentralized Kernel Learning via Linearized ADMM

Quantized Kernel Lleast Lncosh Algorithm

A Quantized Kernel Learning Algorithm Using a Minimum Kernel Risk-Sensitive Loss Criterion and Bilateral Gradient Technique.

Sparse Kernel Recursive Least Squares Using L1 Regularization and a Fixed-Point Sub-Iteration

Fixed budget quantized kernel least-mean-square algorithm

Kernel Least Mean Square Based on Conjugate Gradient.

A Robust Kernel Least Mean Square Algorithm and Its Quantization

ℓq-Norm Sample-Adaptive Multiple Kernel Learning

Revisiting $L_q(0\leq q<1)$ Norm Regularized Optimization

Fast Learning with Nonconvex L1-2 Regularization.

Kernel Regression with Coefficient-based l(q)-regularization

Simplified Quantised Kernel Least Mean Square Algorithm with Fixed Budget