Abstract:Nonnegative matrix factorization (NMF) is a powerful matrix decomposition technique that approximates a nonnegative matrix by the product of two low-rank nonnegative matrix factors. It has been widely applied to signal processing, computer vision, and data mining. Traditional NMF solvers include the multiplicative update rule (MUR), the projected gradient method (PG), the projected nonnegative least squares (PNLS), and the active set method (AS). However, they suffer from one or some of the following three problems: slow convergence rate, numerical instability and nonconvergence. In this paper, we present a new efficient NeNMF solver to simultaneously overcome the aforementioned problems. It applies Nesterov's optimal gradient method to alternatively optimize one factor with another fixed. In particular, at each iteration round, the matrix factor is updated by using the PG method performed on a smartly chosen search point, where the step size is determined by the Lipschitz constant. Since NeNMF does not use the time consuming line search and converges optimally at rate O(1/k(2)) in optimizing each matrix factor, it is superior to MUR and PG in terms of efficiency as well as approximation accuracy. Compared to PNLS and AS that suffer from numerical instability problem in the worst case, NeNMF overcomes this deficiency. In addition, NeNMF can be used to solve L-1-norm, L-2-norm and manifold regularized NMF with the optimal convergence rate. Numerical experiments on both synthetic and real-world datasets show the efficiency of NeNMF for NMF and its variants comparing to representative NMF solvers. Extensive experiments on document clustering suggest the effectiveness of NeNMF.

A Gradient-Based Algorithm with Nonmonotone Line Search for Nonnegative Matrix Factorization

An Adaptive Nonmonotone Projected Barzilai-Borwein Gradient Method with Active Set Prediction for Nonnegative Matrix Factorization

A Two-Point Step Size Gradient Method Fornon-Negative Matrix Factorization

Non-monotone projection gradient method for non-negative matrix factorization

The convergence of conjugate gradient method with nonmonotone line search

NeNMF: an Optimal Gradient Method for Nonnegative Matrix Factorization.

Modified Subspace Barzilai-Borwein Gradient Method for Non-Negative Matrix Factorization

A Fast Algorithm for Nonnegative Matrix Factorization and Its Convergence.

Barzilai and Borwein conjugate gradient method equipped with a non-monotone line search technique and its application on non-negative matrix factorization

EFFICIENT NONNEGATIVE MATRIX FACTORIZATION VIA MODIFIED MONOTONE BARZILAI-BORWEIN METHOD WITH ADAPTIVE STEP SIZES STRATEGY

Fastnmf: A Fast Monotonic Fixed-Point Non-Negative Matrix Factorization Algorithm With High Ease Of Use

A Sparse Algorithm for Non-Negative Matrix Factorization Based on Projection Gradient Method

Accelerated Algorithm to Incomplete Nonnegative Matrix Factorization

FastNMF: Highly Efficient Monotonic Fixed-Point Nonnegative Matrix Factorization Algorithm with Good Applicability

Large-Cone Nonnegative Matrix Factorization

Fast Hierarchical Alternating Nonnegative Least Squares Algorithmfor Nonnegative Matrix Factorization

A New Nonmonotone Spectral Projected Gradient Algorithm for Box-Constrained Optimization Problems in M×n Real Matrix Space with Application in Image Clustering.

A New Active Set Method For Nonnegative Matrix Factorization

Optimization and expansion of non-negative matrix factorization

A Symmetric Rank-one Quasi Newton Method for Non-negative Matrix Factorization