Abstract:Sparse optimization refers to an optimization problem involving the zero-norm in objective or constraints. In this paper, nonconvex approximation approaches for sparse optimization have been studied with a unifying point of view in DC (Difference of Convex functions) programming framework. Considering a common DC approximation of the zero-norm including all standard sparse inducing penalty functions, we studied the consistency between global minimums (resp. local minimums) of approximate and original problems. We showed that, in several cases, some global minimizers (resp. local minimizers) of the approximate problem are also those of the original problem. Using exact penalty techniques in DC programming, we proved stronger results for some particular approximations, namely, the approximate problem, with suitable parameters, is equivalent to the original problem. The efficiency of several sparse inducing penalty functions have been fully analyzed. Four DCA (DC Algorithm) schemes were developed that cover all standard algorithms in nonconvex sparse approximation approaches as special versions. They can be viewed as, an ℓ1-perturbed algorithm/reweighted-ℓ1 algorithm / reweighted-ℓ2 algorithm. We offer a unifying nonconvex approximation approach, with solid theoretical tools as well as efficient algorithms based on DC programming and DCA, to tackle the zero-norm and sparse optimization. As an application, we implemented our methods for the feature selection in SVM (Support Vector Machine) problem and performed empirical comparative numerical experiments on the proposed algorithms with various approximation functions.

An efficient asymptotic DC method for sparse and low-rank matrix recovery

Convergence Analysis of Projected Gradient Descent for Schatten-p Nonconvex Matrix Recovery

CONVERGENCE AND STABILITY OF ITERATIVELY REWEIGHTED LEAST SQUARES FOR LOW-RANK MATRIXRECOVERY

An inexact proximal DC algorithm with sieving strategy for rank constrained least squares semidefinite programming

Nonconvex relaxation approaches to robust matrix recovery

SPARSE AND LOW-RANK MATRIX DECOMPOSITION VIA ALTERNATING DIRECTION METHOD

An Alternating Direction Method with Continuation for Nonconvex Low Rank Minimization

DC approximation approaches for sparse optimization

Approximation Conjugate Gradient Method for Low-Rank Matrix Recovery

An Efficient Semismooth Newton Method for Adaptive Sparse Signal Recovery Problems

Low-rank matrix recovery with non-quadratic loss: projected gradient method and regularity projection oracle

A randomized sparse Kaczmarz solver for sparse signal recovery via minimax‐concave penalty

A Non-convex Approach for Sparse Recovery with Convergence Guarantee

A refined convergence analysis of pDCA$_e$ with applications to simultaneous sparse recovery and outlier detection

Fast and Robust Spectrally Sparse Signal Recovery: A Provable Non-Convex Approach via Robust Low-Rank Hankel Matrix Reconstruction

A NONCONVEX ALGORITHM FOR SIMULTANEOUSLY SPARSE AND LOW-RANK MATRIX RECONSTRUCTION

A Fast Convex Conjugated Algorithm for Sparse Recovery

Unified Analysis on L1 over L2 Minimization for Signal Recovery

A randomized algorithm for simultaneously diagonalizing symmetric matrices by congruence

Fast Hybrid Algorithm for Big Matrix Recovery

Efficient DC Algorithm for Constrained Sparse Optimization