Abstract:Nonnegative matrix factorization (NMF) is a powerful matrix decomposition technique that approximates a nonnegative matrix by the product of two low-rank nonnegative matrix factors. It has been widely applied to signal processing, computer vision, and data mining. Traditional NMF solvers include the multiplicative update rule (MUR), the projected gradient method (PG), the projected nonnegative least squares (PNLS), and the active set method (AS). However, they suffer from one or some of the following three problems: slow convergence rate, numerical instability and nonconvergence. In this paper, we present a new efficient NeNMF solver to simultaneously overcome the aforementioned problems. It applies Nesterov's optimal gradient method to alternatively optimize one factor with another fixed. In particular, at each iteration round, the matrix factor is updated by using the PG method performed on a smartly chosen search point, where the step size is determined by the Lipschitz constant. Since NeNMF does not use the time consuming line search and converges optimally at rate O(1/k(2)) in optimizing each matrix factor, it is superior to MUR and PG in terms of efficiency as well as approximation accuracy. Compared to PNLS and AS that suffer from numerical instability problem in the worst case, NeNMF overcomes this deficiency. In addition, NeNMF can be used to solve L-1-norm, L-2-norm and manifold regularized NMF with the optimal convergence rate. Numerical experiments on both synthetic and real-world datasets show the efficiency of NeNMF for NMF and its variants comparing to representative NMF solvers. Extensive experiments on document clustering suggest the effectiveness of NeNMF.

Neuc-MDS: Non-Euclidean Multidimensional Scaling Through Bilinear Forms

Multi-dimensional Scaling from K-Nearest Neighbourhood Distances

A dual basis approach to multidimensional scaling

Multidimensional Scaling on Multiple Input Distance Matrices

Multidimensional scaling for big data

Continuous Multidimensional Scaling

Bayesian Hyperbolic Multidimensional Scaling

Convex optimization learning of faithful Euclidean distance representations in nonlinear dimensionality reduction

Non-Negative Matrix Factorization with Sinkhorn Distance.

Neural Bregman Divergences for Distance Learning

Modern Multidimensional Scaling: Theory and Applications

NeNMF: an Optimal Gradient Method for Nonnegative Matrix Factorization.

A Supplement to Multidimensional Scaling Framework Formobile Location: a Unified View.

SQuadMDS: a lean Stochastic Quartet MDS improving global structure preservation in neighbor embedding like t-SNE and UMAP

A Globally Convergent Inertial First-Order Optimization Method for Multidimensional Scaling

RMLR: Extending Multinomial Logistic Regression into General Geometries

Neighbourhood-preserving Dimension Reduction Via Localised Multidimensional Scaling

A Divide-and-Conquer Based Multidimensional Scaling Algorithm

Heterogeneous Multidimensional Scaling For Complex Networks

Subspace Clustering Guided Convex Nonnegative Matrix Factorization.

DIMAL: Deep Isometric Manifold Learning Using Sparse Geodesic Sampling