Abstract:Sparsity finds applications in diverse areas such as statistics, machine learning, and signal processing. Computations over sparse structures are less complex compared to their dense counterparts and need less storage. This paper proposes a heuristic method for retrieving sparse approximate solutions of optimization problems via minimizing the quasi-norm, where \(0<p<1\) . An iterative two-block algorithm for minimizing the quasi-norm subject to convex constraints is proposed. The proposed algorithm requires solving for the roots of a scalar degree polynomial as opposed to applying a soft thresholding operator in the case of norm minimization. The algorithm's merit relies on its ability to solve the quasi-norm minimization subject to any convex constraints set. For the specific case of constraints defined by differentiable functions with Lipschitz continuous gradient, a second, faster algorithm is proposed. Using a proximal gradient step, we mitigate the convex projection step and hence enhance the algorithm's speed while proving its convergence. We present various applications where the proposed algorithm excels, namely, sparse signal reconstruction, system identification, and matrix completion. The results demonstrate the significant gains obtained by the proposed algorithm compared to other quasi-norm based methods presented in previous literature.</p<1\)<>

Deflation Techniques for Finding Multiple Local Minima of a Nonlinear Least Squares Problem

Continuation Newton methods with deflation techniques for global optimization problems

Deflated Iterative Methods for Linear Equations with Multiple Right-Hand Sides

Deflation for the off-diagonal block in symmetric saddle point systems

Two-step Newton's method for deflation-one singular zeros of analytic systems

KRYLOV SUBSPACE METHODS WITH DEFLATION AND BALANCING PRECONDITIONERS FOR LEAST SQUARES PROBLEMS

Two-Grid Deflated Krylov Methods for Linear Equations

A Multilevel Low-Rank Newton Method with Super-linear Convergence Rate and its Application to Non-convex Problems

Yet another fast variant of Newton’s method for nonconvex optimization

Scalable Subspace Methods for Derivative-Free Nonlinear Least-Squares Optimization

Inexact Newton-type Methods for Optimisation with Nonnegativity Constraints

A Deflated Block Flexible Gmres-Dr Method for Linear Systems with Multiple Right-Hand Sides

On a Family of Relaxed Gradient Descent Methods for Quadratic Minimization

Gauss-Southwell type descent methods for low-rank matrix optimization

Lp quasi-norm minimization: algorithm and applications

A Fast Scale-Invariant Algorithm for Non-negative Least Squares with Non-negative Data

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes

On the saddle point problem for non-convex optimization

Certifying the absence of spurious local minima at infinity