Abstract:We present a fast, efficient algorithm for learning an overcomplete dictionary for sparse representation of signals. The whole problem is considered as a minimization of the approximation error function with a coherence penalty for the dictionary atoms and with the sparsity regularization of the coefficient matrix. Because the problem is nonconvex and nonsmooth, this minimization problem cannot be solved efficiently by an ordinary optimization method. We propose a decomposition scheme and an alternating optimization that can turn the problem into a set of minimizations of piecewise quadratic and univariate subproblems, each of which is a single variable vector problem, of either one dictionary atom or one coefficient vector. Although the subproblems are still nonsmooth, remarkably they become much simpler so that we can find a closed-form solution by introducing a proximal operator. This leads to an efficient algorithm for sparse representation. To our knowledge, applying the proximal operator to the problem with an incoherence term and obtaining the optimal dictionary atoms in closed form with a proximal operator technique have not previously been studied. The main advantages of the proposed algorithm are that, as suggested by our analysis and simulation study, it has lower computational complexity and a higher convergence rate than state-of-the-art algorithms. In addition, for real applications, it shows good performance and significant reductions in computational time.

An Efficient Algorithm for Dictionary Learning with a Mixed-Norm Regularizer for Sparsity Based on Proximal Operator.

Dictionary Learning with ℓ1/2 Regularizer for Sparsity Based on Proximal Operator

Dictionary Learning with &#x2113;<inf>1/2</inf> Regularizer for Sparsity Based on Proximal Operator

Dictionary Learning With Log-Regularizer For Sparse Representation

Dictionary Learning for Sparse Representation Using Weighted ℓ<inf>1</inf>-Norm

A Fast Algorithm for Learning Overcomplete Dictionary for Sparse Representation Based on Proximal Operators

Manifold Optimization-Based Analysis Dictionary Learning with an ℓ1∕2-Norm Regularizer.

Dictionary learning for sparse representation using weighted ℓ<inf>1</inf>-norm

A Novel Dictionary Learning Method for Sparse Representation with Nonconvex Regularizations.

Constrained Analysis Dictionary Learning with the L(1/2)-Norm Regularizer

Dictionary Learning with the $${{\ell }_{1/2}}$$-Regularizer and the Coherence Penalty and Its Convergence Analysis

Incoherent dictionary learning with log-regularizer based on proximal operators.

An Efficient Orthonormalization-Free Approach for Sparse Dictionary Learning and Dual Principal Component Pursuit

Complete Dictionary Learning Via L4-Norm Maximization over the Orthogonal Group

A Non-Convex Relaxation Approach to Sparse Dictionary Learning

Analysis Dictionary Learning Using Block Coordinate Descent Framework with Proximal Operators

A Fast Holistic Algorithm for Complete Dictionary Learning via ` Norm Maximization

Structural Dictionary Learning Based on Non-convex Surrogate of ℓ₂, ₁ Norm for Classification.

L0 Norm Based Dictionary Learning by Proximal Methods with Global Convergence

Dictionary learning under global sparsity constraint

Complete Dictionary Learning via $\ell^4$-Norm Maximization over the Orthogonal Group