Abstract:We investigate a class of constrained sparse regression problem with cardinality penalty, where the feasible set is defined by box constraint, and the loss function is convex, but not necessarily smooth. First, we put forward a smoothing fast iterative hard thresholding (SFIHT) algorithm for solving such optimization problems, which combines smoothing approximations, extrapolation techniques and iterative hard thresholding methods. The extrapolation coefficients can be chosen to satisfy $\sup_k \beta_k=1$ in the proposed algorithm. We discuss the convergence behavior of the algorithm with different extrapolation coefficients, and give sufficient conditions to ensure that any accumulation point of the iterates is a local minimizer of the original cardinality penalized problem. In particular, for a class of fixed extrapolation coefficients, we discuss several different update rules of the smoothing parameter and obtain the convergence rate of $O(\ln k/k)$ on the loss and objective function values. Second, we consider the case in which the loss function is Lipschitz continuously differentiable, and develop a fast iterative hard thresholding (FIHT) algorithm to solve it. We prove that the iterates of FIHT converge to a local minimizer of the problem that satisfies a desirable lower bound property. Moreover, we show that the convergence rate of loss and objective function values are $o(k^{-2})$. Finally, some numerical examples are presented to illustrate the theoretical results.

Hard Thresholding Pursuit with Continuation for ℓ0‐regularized Minimizations

L0 image smoothing via iterating truncated L1 gradient regularization

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

A New Linear Convergence Result for the Iterative Soft Thresholding Algorithm

Iterative thresholding algorithm based on non-convex method for modified lp-norm regularization minimization

A Primal Dual Active Set with Continuation Algorithm for the ℓ^0-Regularized Optimization Problem

Psychosomatic digitalis-toxic arrhythmias in guinea pigs.

Exact Recovery of Hard Thresholding Pursuit.

Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization.

Smoothing fast iterative hard thresholding algorithm for $\ell_0$ regularized nonsmooth convex regression problem

A splitting primal-dual proximity algorithm for solving composite optimization problems

Fast Learning with Nonconvex L1-2 Regularization.

Scaled Proximal Gradient Methods for Sparse Optimization Problems

An Iterative Threshold Algorithm of Log-sum Regularization for Sparse Problem

Minimizing L 1 over L 2 norms on the gradient

On the Convergence of an Active-Set Method for ℓ1minimization

Revisiting $L_q(0\leq q<1)$ Norm Regularized Optimization

The variant of the iterative shrinkage-thresholding algorithm for minimization of the l 1 over l ∞ norms

A new inexact iterative hard thresholding algorithm for compressed sensing

New Inexact Explicit Thresholding/shrinkage Formulas for Inverse Problems with Overlapping Group Sparsity.

A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem