Abstract:Graph regularized nonnegative matrix factorization (GNMF) decomposes a nonnegative data matrix X[Symbol:see text]R(m x n) to the product of two lower-rank nonnegative factor matrices, i.e.,W[Symbol:see text]R(m x r) and H[Symbol:see text]R(r x n) (r < min {m,n}) and aims to preserve the local geometric structure of the dataset by minimizing squared Euclidean distance or Kullback-Leibler (KL) divergence between X and WH. The multiplicative update rule (MUR) is usually applied to optimize GNMF, but it suffers from the drawback of slow-convergence because it intrinsically advances one step along the rescaled negative gradient direction with a non-optimal step size. Recently, a multiple step-sizes fast gradient descent (MFGD) method has been proposed for optimizing NMF which accelerates MUR by searching the optimal step-size along the rescaled negative gradient direction with Newton's method. However, the computational cost of MFGD is high because 1) the high-dimensional Hessian matrix is dense and costs too much memory; and 2) the Hessian inverse operator and its multiplication with gradient cost too much time. To overcome these deficiencies of MFGD, we propose an efficient limited-memory FGD (L-FGD) method for optimizing GNMF. In particular, we apply the limited-memory BFGS (L-BFGS) method to directly approximate the multiplication of the inverse Hessian and the gradient for searching the optimal step size in MFGD. The preliminary results on real-world datasets show that L-FGD is more efficient than both MFGD and MUR. To evaluate the effectiveness of L-FGD, we validate its clustering performance for optimizing KL-divergence based GNMF on two popular face image datasets including ORL and PIE and two text corpora including Reuters and TDT2. The experimental results confirm the effectiveness of L-FGD by comparing it with the representative GNMF solvers.

Limited-memory Fast Gradient Descent Method for Graph Regularized Nonnegative Matrix Factorization.

Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent.

Distributed Graph Regularized Non-Negative Matrix Factorization with Greedy Coordinate Descent.

NeNMF: an Optimal Gradient Method for Nonnegative Matrix Factorization.

Online Graph Regularized Non-Negative Matrix Factorization for Large-Scale Datasets

Efficient Rank-one Residue Approximation Method for Graph Regularized Non-negative Matrix Factorization.

Flexible Non-Negative Matrix Factorization With Adaptively Learned Graph Regularization

Graph Regularized Sparse Non-Negative Matrix Factorization for Clustering

Local coordinate based graph-regularized NMF for image representation

Graph Regularized Discriminative Non-Negative Matrix Factorization for Face Recognition

Convergence Analysis of Graph Regularized Non-Negative Matrix Factorization

Adaptive graph regularized nonnegative matrix factorization for data representation

Feature Selection and Multi-Kernel Learning for Adaptive Graph Regularized Nonnegative Matrix Factorization.

Non-negative Low-Rank and Group-Sparse Matrix Factorization

Graph Regularized Nonnegative Matrix Factorization with Sample Diversity for Image Representation.

Graph Regularized NMF with L20-norm for Unsupervised Feature Learning

A Generalized Deep Learning Clustering Algorithm Based on Non-Negative Matrix Factorization

Dual Graph Regularized Sparse Nonnegative Matrix Factorization For Data Representation

Graph Regularized Nonnegative Matrix Factorization with Sparse Coding

Robust Semi-Supervised Non-Negative Matrix Factorization with Structured Normalization.

Graph regularized non-negative matrix factorization with sparse coding