Abstract:Dictionary Learning (DL) plays a crucial role in numerous machine learning tasks. It targets at finding the dictionary over which the training set admits a maximally sparse representation. Most existing DL algorithms are based on solving an optimization problem, where the noise variance and sparsity level should be known as the prior knowledge. However, in practice applications, it is difficult to obtain these knowledge. Thus, non-parametric Bayesian DL has recently received much attention of researchers due to its adaptability and effectiveness. Although many hierarchical priors have been used to promote the sparsity of the representation in non-parametric Bayesian DL, the problem of redundancy for the dictionary is still overlooked, which greatly decreases the performance of sparse coding. To address this problem, this paper presents a novel robust dictionary learning framework via Bayesian inference. In particular, we employ the orthogonality-promoting regularization to mitigate correlations among dictionary atoms. Such a regularization, encouraging the dictionary atoms to be close to being orthogonal, can alleviate overfitting to training data and improve the discrimination of the model. Moreover, we impose Scale mixture of the Vector variate Gaussian (SMVG) distribution on the noise to capture its structure. A Regularized Expectation Maximization Algorithm is developed to estimate the posterior distribution of the representation and dictionary with orthogonality-promoting regularization. Numerical results show that our method can learn the dictionary with an accuracy better than existing methods, especially when the number of training signals is limited.

A Fast Holistic Algorithm for Complete Dictionary Learning via ` Norm Maximization

Complete Dictionary Learning Via L4-Norm Maximization over the Orthogonal Group

Complete Dictionary Learning via $\ell^4$-Norm Maximization over the Orthogonal Group

Understanding L4-Based Dictionary Learning: Interpretation, Stability, and Robustness.

An Efficient Algorithm for Dictionary Learning with a Mixed-Norm Regularizer for Sparsity Based on Proximal Operator.

Decentralized Complete Dictionary Learning via $\ell^{4}$-Norm Maximization

Manifold Optimization-Based Analysis Dictionary Learning with an ℓ1∕2-Norm Regularizer.

An Efficient Orthonormalization-Free Approach for Sparse Dictionary Learning and Dual Principal Component Pursuit

Dictionary Learning for Sparse Representation Using Weighted ℓ<inf>1</inf>-Norm

Dictionary learning for sparse representation using weighted ℓ<inf>1</inf>-norm

Simple Alternating Minimization Provably Solves Complete Dictionary Learning

Orthogonality-Promoting Dictionary Learning Via Bayesian Inference.

A Fast Algorithm for Learning Overcomplete Dictionary for Sparse Representation Based on Proximal Operators

L0 Norm Based Dictionary Learning by Proximal Methods with Global Convergence

Dictionary Learning with &#x2113;<inf>1/2</inf> Regularizer for Sparsity Based on Proximal Operator

A Novel Dictionary Learning Method for Sparse Representation with Nonconvex Regularizations.

Dictionary Learning with ℓ1/2 Regularizer for Sparsity Based on Proximal Operator

Proximal Alternating Minimization for Analysis Dictionary Learning and Convergence Analysis

Dictionary learning under global sparsity constraint

Convergence Analysis of Kernel LMS Algorithm with Pre-Tuned Dictionary

Overcomplete Dictionary Learning For Nonnegative Sparse Representation With An (Lp)-Norm Constraint Based On Majorize-Minimization