Abstract:Non-negative matrix factorization (NMF) is a powerful machine learning method for decomposing a high-dimensional nonnegative matrix V into the product of two nonnegative matrices, W and H, such that V ∼ W H. It has been shown to have a parts-based, sparse representation of the data. NMF has been successfully applied in a variety of areas such as natural language processing, neuroscience, information retrieval, image processing, speech recognition and computational biology for the analysis and interpretation of large-scale data. There has also been simultaneous development of a related statistical latent class modeling approach, namely, probabilistic latent semantic indexing (PLSI), for analyzing and interpreting co-occurrence count data arising in natural language processing. In this paper, we present a generalized statistical approach to NMF and PLSI based on Renyi's divergence between two non-negative matrices, stemming from the Poisson likelihood. Our approach unifies various competing models and provides a unique theoretical framework for these methods. We propose a unified algorithm for NMF and provide a rigorous proof of monotonicity of multiplicative updates for W and H. In addition, we generalize the relationship between NMF and PLSI within this framework. We demonstrate the applicability and utility of our approach as well as its superior performance relative to existing methods using real-life and simulated document clustering data.

Variational Regularized 2-D Nonnegative Matrix Factorization.

Minimum Variance Block-Based Nonnegative Matrix Factorization Algorithm for Hyperspectral Unmixing.

A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization

Robust Bayesian Nonnegative Matrix Factorization with Implicit Regularizers

Efficient algorithms for regularized Poisson Non-negative Matrix Factorization

Adversarially-Trained Nonnegative Matrix Factorization

Maximum Discrepancy Generative Regularization and Non-Negative Matrix Factorization for Single Channel Source Separation

Graph Regularized Sparse $L_{2,1}$ Semi-Nonnegative Matrix Factorization for Data Reduction

Graph Regularized Sparse L2,1 Semi‐Nonnegative Matrix Factorization for Data Reduction

Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization

Comparative Study of Inference Methods for Bayesian Nonnegative Matrix Factorisation

A unified statistical approach to non-negative matrix factorization and probabilistic latent semantic indexing

Locality Preserving Nonnegative Matrix Factorization.

Diversified Bayesian Nonnegative Matrix Factorization

Automatic Relevance Determination in Nonnegative Matrix Factorization with the β-Divergence

A statistical framework for non-negative matrix factorization based on generalized dual divergence

Graph regularized non-negative matrix factorization with sparse coding

Fast Parallel Randomized Algorithm for Nonnegative Matrix Factorization with KL Divergence for Large Sparse Datasets

Robust Asymmetric Bayesian Adaptive Matrix Factorization

Spatially correlated nonnegative matrix factorization

Robust Matrix Factorization by Majorization Minimization