Abstract:Dictionary Learning (DL) plays a crucial role in numerous machine learning tasks. It targets at finding the dictionary over which the training set admits a maximally sparse representation. Most existing DL algorithms are based on solving an optimization problem, where the noise variance and sparsity level should be known as the prior knowledge. However, in practice applications, it is difficult to obtain these knowledge. Thus, non-parametric Bayesian DL has recently received much attention of researchers due to its adaptability and effectiveness. Although many hierarchical priors have been used to promote the sparsity of the representation in non-parametric Bayesian DL, the problem of redundancy for the dictionary is still overlooked, which greatly decreases the performance of sparse coding. To address this problem, this paper presents a novel robust dictionary learning framework via Bayesian inference. In particular, we employ the orthogonality-promoting regularization to mitigate correlations among dictionary atoms. Such a regularization, encouraging the dictionary atoms to be close to being orthogonal, can alleviate overfitting to training data and improve the discrimination of the model. Moreover, we impose Scale mixture of the Vector variate Gaussian (SMVG) distribution on the noise to capture its structure. A Regularized Expectation Maximization Algorithm is developed to estimate the posterior distribution of the representation and dictionary with orthogonality-promoting regularization. Numerical results show that our method can learn the dictionary with an accuracy better than existing methods, especially when the number of training signals is limited.

Sparse Representation over Overcomplete Dictionary Based on Bayesian Nonparametric Approximation

Image Sparse Representation Based on A Nonparametric Bayesian Model

A Fast Algorithm for Learning Overcomplete Dictionary for Sparse Representation Based on Proximal Operators

A Nonparametric Bayesian Approach To Joint Multiple Dictionary Learning With Separate Image Sources

Sparse Bayesian Dictionary Learning with a Gaussian Hierarchical Model.

Sparse Signal Recovery Via Non-Convex Optimization and Overcomplete Dictionaries.

Nonnegative Sparse Representation Based on the Determinant Measure

Dictionary Learning for Sparse Representation Using Weighted ℓ<inf>1</inf>-Norm

Learning dictionary from signals under global sparsity constraint

Dictionary Learning With Log-Regularizer For Sparse Representation

An overcomplete dictionary design algorithm for sparse representation of piecewise stationary signals

Forward Basis Selection for Sparse Approximation over Dictionary

Non-parametric Bayesian Dictionary Learning Based on Laplace Noise

Bayesian nonparametric dictionary learning for compressed sensing MRI.

Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization

An Efficient Dictionary Learning Algorithm for Sparse Representation

Sparsity and Low-Rank Dictionary Learning for Sparse Representation of Monogenic Signal

A New Method for Signal Sparse Decomposition

Dictionary learning for sparse representation using weighted ℓ<inf>1</inf>-norm

Bayesian sparsity and class sparsity priors for dictionary learning and coding

Orthogonality-Promoting Dictionary Learning Via Bayesian Inference.