Abstract:Nonnegative matrix factorization (NMF) is a linear dimensionality technique for nonnegative data with applications such as image analysis, text mining, audio source separation and hyperspectral unmixing. Given a data matrix $M$ and a factorization rank $r$, NMF looks for a nonnegative matrix $W$ with $r$ columns and a nonnegative matrix $H$ with $r$ rows such that $M \approx WH$. NMF is NP-hard to solve in general. However, it can be computed efficiently under the separability assumption which requires that the basis vectors appear as data points, that is, that there exists an index set $\mathcal{K}$ such that $W = M(:,\mathcal{K})$. In this paper, we generalize the separability assumption: We only require that for each rank-one factor $W(:,k)H(k,:)$ for $k=1,2,\dots,r$, either $W(:,k) = M(:,j)$ for some $j$ or $H(k,:) = M(i,:)$ for some $i$. We refer to the corresponding problem as generalized separable NMF (GS-NMF). We discuss some properties of GS-NMF and propose a convex optimization model which we solve using a fast gradient method. We also propose a heuristic algorithm inspired by the successive projection algorithm. To verify the effectiveness of our methods, we compare them with several state-of-the-art separable NMF algorithms on synthetic, document and image data sets.
Machine Learning,Computer Vision and Pattern Recognition,Optimization and Control
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: in non - negative matrix factorization (NMF), when the data matrix \( \mathbf{M} \) satisfies the generalized separability assumption, how to efficiently find its approximate decomposition. Specifically, the paper proposes the Generalized Separable Nonnegative Matrix Factorization (GS - NMF) and attempts to solve the following problems:
1. **Limitations of traditional NMF**: Standard NMF is an NP - hard problem, and the solutions are usually not unique. To solve these problems, the separability assumption was introduced, but this assumption is sometimes too strict in practical applications.
2. **Generalized separability assumption**: The paper relaxes the traditional separability assumption and proposes a more flexible generalized separability assumption. According to this assumption, for each rank - 1 factor \( \mathbf{W}(:,k)\mathbf{H}(k,:) \), either \( \mathbf{W}(:,k)=\mathbf{M}(:,j) \) for some \( j \), or \( \mathbf{H}(k,:)=\mathbf{M}(i,:) \) for some \( i \).
3. **Algorithm design and optimization model**: To deal with the generalized separable NMF problem, the paper proposes a convex optimization model and uses the fast gradient method to solve this model. In addition, a heuristic algorithm inspired by the successful projection algorithm is also proposed.
4. **Experimental verification**: Through experiments on synthetic data sets, document data sets, and image data sets, the effectiveness of the proposed GS - NMF method is verified, especially in low - rank approximation and error control, which is superior to the traditional separable NMF algorithm.
### Mathematical formula summary
- **Definition of generalized separability**:
\[
\mathbf{M}=\mathbf{M}(:,K_1)\mathbf{P}_1+\mathbf{P}_2\mathbf{M}(K_2,:)
\]
where \( K_1 \) and \( K_2 \) are index sets respectively, \( \mathbf{P}_1\in\mathbb{R}^{r_1\times n}_+ \) and \( \mathbf{P}_2\in\mathbb{R}^{m\times r_2}_+ \) are non - negative matrices, satisfying \( \mathbf{P}_1(:,K_1)=I_{r_1} \) and \( \mathbf{P}_2(K_2,:)=I_{r_2} \).
- **Optimization model**:
\[
\min_{\mathbf{X}\in\mathbb{R}^{n\times n}_+,\mathbf{Y}\in\mathbb{R}^{m\times m}_+}\text{trace}(\mathbf{X})+\text{trace}(\mathbf{Y})
\]
such that
\[
\|\mathbf{M}-\mathbf{M}\mathbf{X}-\mathbf{Y}\mathbf{M}\|\leq\epsilon,
\]
and
\[
\mathbf{X}(i,j)\leq\mathbf{X}(i,i)\leq1\quad\text{for}\quad1\leq i,j\leq n,
\]
\[
\mathbf{Y}(l,t)\leq\mathbf{Y}(t,t)\leq1\quad\text{for}\quad1\leq l,t\leq m.
\]
Through these improvements, the paper aims to provide a more effective tool to handle large - scale non - negative matrix factorization problems and show its superior performance in multiple practical application scenarios.