Abstract:We consider the problem of hierarchical sparse coding, where not only a few groups of atoms are active at a time but also each group enjoys internal sparsity. The current approaches are usually to achieve between-group sparsity using the ℓ1 penalty, such that many groups have small coefficients rather than being accurately zeroed out. The trivial groups may incur the proneness to overfitting of noise and are thereby harmful to interpretability of sparse representation. To this end, we in this paper reformulate the hierarchical sparse model from a Bayesian perspective employing twofold priors: the spike-and-slab prior and the Laplacian prior. The former is utilized to explicitly induce between-group sparsity, while the latter is adopted for both inducing within-group sparsity and obtaining a small reconstruction error. We propose a nest prior by integrating the both priors to result in hierarchical sparsity. The resultant optimization problem can be delivered a convergence solution in a few iterations via the proposed nested algorithm, corresponding to the nested prior. In experiments, we evaluate the performance of our method on signal recovery, image inpainting and sparse representation based classification, with simulated signals and two publicly available image databases. The results manifest that the proposed method, compared with the popular methods for sparse coding, can yield more concise representation and more reliable interpretation of data.

Learning Word Representations with Hierarchical Sparse Coding

Sparse Overcomplete Word Vector Representations

Learning Sparse Overcomplete Word Vectors Without Intermediate Dense Representations

Word Equations: Inherently Interpretable Sparse Word Embeddingsthrough Sparse Coding

Interpretable Syntactic Representations Enable Hierarchical Word Vectors

Hierarchical Sparse Coding from a Bayesian Perspective.

Sparse Poisson coding for high dimensional document clustering

Learning Sparse, Distributed Representations using the Hebbian Principle

Parallel And Hierarchical Decision Making For Sparse Coding In Speech Recognition

Biologically Plausible Sparse Temporal Word Representations

Learning Topic-Sensitive Word Representations

Hierarchical Sparse Coding for Visual Co occurrence Discovery

Hierarchical Sparse Method with Applications in Vision and Speech Recognition

Sparse Lifting of Dense Vectors: Unifying Word and Sentence Representations

A Hierarchical Latent Topic Model Based on Sparse Coding

Learning Sparse Representations in Reinforcement Learning with Sparse Coding

Sparse word embeddings using l1 regularized online learning

Joint Word Representation Learning using a Corpus and a Semantic Lexicon

Learning Ordered Word Representations

Multimodal sparse representation learning and applications

Unified Dictionary Learning and Region Tagging with Hierarchical Sparse Representation