Abstract:Non-negative matrix factorization (NMF) approximates a non-negative matrix $X$ by a product of two non-negative low-rank factor matrices $W$ and $H$. NMF and its extensions minimize either the Kullback-Leibler divergence or the Euclidean distance between $X$ and $W^T H$ to model the Poisson noise or the Gaussian noise. In practice, when the noise distribution is heavy tailed, they cannot perform well. This paper presents Manhattan NMF (MahNMF) which minimizes the Manhattan distance between $X$ and $W^T H$ for modeling the heavy tailed Laplacian noise. Similar to sparse and low-rank matrix decompositions, MahNMF robustly estimates the low-rank part and the sparse part of a non-negative matrix and thus performs effectively when data are contaminated by outliers. We extend MahNMF for various practical applications by developing box-constrained MahNMF, manifold regularized MahNMF, group sparse MahNMF, elastic net inducing MahNMF, and symmetric MahNMF. The major contribution of this paper lies in two fast optimization algorithms for MahNMF and its extensions: the rank-one residual iteration (RRI) method and Nesterov's smoothing method. In particular, by approximating the residual matrix by the outer product of one row of W and one row of $H$ in MahNMF, we develop an RRI method to iteratively update each variable of $W$ and $H$ in a closed form solution. Although RRI is efficient for small scale MahNMF and some of its extensions, it is neither scalable to large scale matrices nor flexible enough to optimize all MahNMF extensions. Since the objective functions of MahNMF and its extensions are neither convex nor smooth, we apply Nesterov's smoothing method to recursively optimize one factor matrix with another matrix fixed. By setting the smoothing parameter inversely proportional to the iteration number, we improve the approximation accuracy iteratively for both MahNMF and its extensions.

Iterative Improvement of the Multiplicative Update NMF Algorithm Using Nature-Inspired Optimization

A fast Multiplicative Updates algorithm for Non-negative Matrix Factorization

Swarm Intelligence For Non-Negative Matrix Factorization

Optimization and expansion of non-negative matrix factorization

Swarm Intelligence For Dimensionality Reduction: How To Improve The Non-Negative Matrix Factorization With Nature-Inspired Optimization Methods

Online Adaptive Passive-Aggressive Methods for Non-Negative Matrix Factorization and Its Applications

Using Population Based Algorithms For Initializing Nonnegative Matrix Factorization

FastNMF: Highly Efficient Monotonic Fixed-Point Nonnegative Matrix Factorization Algorithm with Good Applicability

NeNMF: an Optimal Gradient Method for Nonnegative Matrix Factorization.

Fastnmf: A Fast Monotonic Fixed-Point Non-Negative Matrix Factorization Algorithm With High Ease Of Use

A Class of Manifold Regularized Multiplicative Update Algorithms for Image Clustering

An optimal pairwise merge algorithm improves the quality and consistency of nonnegative matrix factorization

Swarm Intelligence for Dimensionality Reduction

Multiplicative Algorithms for Symmetric Nonnegative Tensor Factorizations and Its Applications.

A three-stage method for batch-based incremental nonnegative matrix factorization

A unified statistical approach to non-negative matrix factorization and probabilistic latent semantic indexing

Factor-Bounded Nonnegative Matrix Factorization

MahNMF: Manhattan Non-negative Matrix Factorization

Chordal-NMF with Riemannian Multiplicative Update

New SVD based initialization strategy for Non-negative Matrix Factorization

Towards a Fairer Non-negative Matrix Factorization