Abstract:Non-negative matrix factorization (NMF) is an intuitive, non-negative, and interpretable approximation method. Canonical NMF approach could derive some basic components to represent original data, while probabilistic NMF approaches try to introduce some reasonable constraints to optimize the canonical NMF model. However, both of them cannot handle ground-truth bases discovering and model order determination problems. In general, the model order of basis matrix needs to be pre-defined. The model order determines the capability and accuracy of data structure discovering. However, how to accurately infer the model order of basis matrix has not been well investigated. In this paper, we propose a method called non-informative hierarchical Bayesian non-negative matrix factorization (NHBNMF) to automatically determine the model order and discover the data structure. They are achieved through hierarchical Bayesian inference model, maximum a posteriori (MAP) criterion, and non-informative parameters. In NHBNMF method, we first introduce a structure with two-level parameters to enable the entire model to approach the distributions of ground-truth bases. Then we use non-informative parameter scheme to eliminate the hyper-parameter to enable automatic searching. Finally, the model order and ground-truth bases are discovered by using MAP criterion and L2-norm selection. The experiments are conducted based on both synthetic and real-world datasets to show the effectiveness of our algorithm. The results demonstrate that our algorithm can accurately estimate the model order and discover the ground-truth bases. Even for the complicated FERET facial dataset, our algorithm still obtained interpretable bases and achieved satisfactory accuracy of the model order estimation.

Effective Mean-Field Inference Method for Nonnegative Boltzmann Machines

Training Restricted Boltzmann Machines with Binary Synapses Using the Bayesian Learning Rule

Bayesian mean-parameterized nonnegative binary matrix factorization

Monotone deep Boltzmann machines

Nonparametric Bayesian Nonnegative Matrix Factorization

Bayesian Nonnegative Matrix Factorization with Dirichlet Process Mixtures

Non-informative hierarchical Bayesian inference for non-negative matrix factorization

Reweighted Nonnegative Least-Mean-Square Algorithm

Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation

Automatic Relevance Determination in Nonnegative Matrix Factorization with the β-Divergence

Advanced Mean Field Theory of Restricted Boltzmann Machine

A unified statistical approach to non-negative matrix factorization and probabilistic latent semantic indexing

Mean-Field Assisted Deep Boltzmann Learning with Probabilistic Computers

Particle Mean Field Variational Bayes

Nonnegative Restricted Boltzmann Machines for Parts-based Representations Discovery and Predictive Model Stabilization

Neural Boltzmann Machines

Inferring effective couplings with Restricted Boltzmann Machines

Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection

Mean field initialization of the Annealed Importance Sampling algorithm for an efficient evaluation of the Partition Function of Restricted Boltzmann Machines

Community Inference With Bayesian Non-Negative Matrix Factorization

Mitigating the Effects of Non-Identifiability on Inference for Bayesian Neural Networks with Latent Variables