Abstract:Graph regularized nonnegative matrix factorization (GNMF) decomposes a nonnegative data matrix X[Symbol:see text]R(m x n) to the product of two lower-rank nonnegative factor matrices, i.e.,W[Symbol:see text]R(m x r) and H[Symbol:see text]R(r x n) (r < min {m,n}) and aims to preserve the local geometric structure of the dataset by minimizing squared Euclidean distance or Kullback-Leibler (KL) divergence between X and WH. The multiplicative update rule (MUR) is usually applied to optimize GNMF, but it suffers from the drawback of slow-convergence because it intrinsically advances one step along the rescaled negative gradient direction with a non-optimal step size. Recently, a multiple step-sizes fast gradient descent (MFGD) method has been proposed for optimizing NMF which accelerates MUR by searching the optimal step-size along the rescaled negative gradient direction with Newton's method. However, the computational cost of MFGD is high because 1) the high-dimensional Hessian matrix is dense and costs too much memory; and 2) the Hessian inverse operator and its multiplication with gradient cost too much time. To overcome these deficiencies of MFGD, we propose an efficient limited-memory FGD (L-FGD) method for optimizing GNMF. In particular, we apply the limited-memory BFGS (L-BFGS) method to directly approximate the multiplication of the inverse Hessian and the gradient for searching the optimal step size in MFGD. The preliminary results on real-world datasets show that L-FGD is more efficient than both MFGD and MUR. To evaluate the effectiveness of L-FGD, we validate its clustering performance for optimizing KL-divergence based GNMF on two popular face image datasets including ORL and PIE and two text corpora including Reuters and TDT2. The experimental results confirm the effectiveness of L-FGD by comparing it with the representative GNMF solvers.

Distributed Graph Regularized Non-Negative Matrix Factorization with Greedy Coordinate Descent.

Limited-memory Fast Gradient Descent Method for Graph Regularized Nonnegative Matrix Factorization.

Graph Regularized Sparse Non-Negative Matrix Factorization for Clustering

Online Graph Regularized Non-Negative Matrix Factorization for Large-Scale Datasets

Adaptive graph regularized nonnegative matrix factorization for data representation

Flexible Non-Negative Matrix Factorization With Adaptively Learned Graph Regularization

Graph Regularized Discriminative Non-Negative Matrix Factorization for Face Recognition

DID: Distributed Incremental Block Coordinate Descent for Nonnegative Matrix Factorization

Graph regularized non-negative matrix factorization with sparse coding

Graph Regularized Nonnegative Matrix Factorization with Sample Diversity for Image Representation.

Improved Hypergraph Regularized Nonnegative Matrix Factorization with Sparse Representation

Deep graph convolution neural network with non-negative matrix factorization for community discovery

Semi-supervised non-negative matrix tri-factorization with adaptive neighbors and block-diagonal learning

Graph Regularized Nonnegative Matrix Factorization with Sparse Coding

Semi-Supervised Graph Regularized Deep NMF with Bi-Orthogonal Constraints for Data Representation

Globality Constrained Adaptive Graph Regularized Non‐negative Matrix Factorization for Data Representation

Correntropy induced metric based graph regularized non-negative matrix factorization

Graph Regularized NMF with L20-norm for Unsupervised Feature Learning

Feature Selection and Multi-Kernel Learning for Adaptive Graph Regularized Nonnegative Matrix Factorization.

Graph-Based Discriminative Nonnegative Matrix Factorization with Label Information

Dual Graph Regularized Sparse Nonnegative Matrix Factorization For Data Representation