A Convergent Algorithm for Bi-orthogonal Nonnegative Matrix Tri-Factorization

Andri Mirzal

DOI: https://doi.org/10.48550/arXiv.1710.11478

2018-11-15

Abstract:A convergent algorithm for nonnegative matrix factorization with orthogonality constraints imposed on both factors is proposed in this paper. This factorization concept was first introduced by Ding et al. with intent to further improve clustering capability of NMF. However, as the original algorithm was developed based on multiplicative update rules, the convergence of the algorithm cannot be guaranteed. In this paper, we utilize the technique presented in our previous work to develop the algorithm and prove that it converges to a stationary point inside the solution space.

Machine Learning

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to solve the **convergence problem of the orthogonal constraint algorithm in non - negative matrix factorization (NMF)**, especially in the case of biorthogonal non - negative matrix trifactorization (BNMtF). Specifically:

1. **Background and motivation**:
- Non - negative matrix factorization (NMF) is a commonly used dimension reduction and feature extraction technique, which is widely used in image processing, text mining and other fields.
- Ding et al. proposed two NMF methods with orthogonal constraints: unitary non - negative matrix factorization (UNMF) and biorthogonal non - negative matrix trifactorization (BNMtF). These methods aim to improve the clustering ability of NMF.
- However, the algorithms based on the multiplicative update rule proposed by Ding et al. cannot guarantee convergence.

2. **Problem description**:
- **Convergence problem**: The existing BNMtF algorithms based on the multiplicative update rule cannot guarantee convergence to a stable point. This means that the algorithm may not gradually optimize the objective function as the number of iterations increases.
- **Limitations of existing algorithms**: Due to the existence of orthogonal constraints, some convergence algorithms of standard NMF (such as algorithms in the alternating non - negative least squares (ANLS) framework) cannot be directly applied to the BNMtF problem.

3. **Research objectives**:
- **Propose a new convergence algorithm**: The author developed a new BNMtF algorithm using the techniques in previous work and proved that this algorithm can converge to a stable point in the solution space.
- **Improve clustering performance**: By introducing regularization parameters α and β to adjust the degree of orthogonality of B and C, further improve the clustering effect while maintaining the orthogonal constraints.

4. **Mathematical formula**:
- The objective function of BNMtF can be expressed as:
\[
\min_{B, C, S} J(B, C, S)=\frac{1}{2}\|A - BSC\|_F^2+\frac{\alpha}{2}\|CCT - I\|_F^2+\frac{\beta}{2}\|BTB - I\|_F^2
\]
where \(A\in\mathbb{R}^{M\times N}_+\) is the data matrix, \(B\in\mathbb{R}^{M\times P}_+\) is the basis matrix, \(C\in\mathbb{R}^{Q\times N}_+\) is the coefficient matrix, \(S\in\mathbb{R}^{P\times Q}_+\) is the third factor for absorbing scaling differences, and α and β are regularization parameters.

5. **Conclusion**:
- The author verified the effectiveness of the new algorithm through experiments, especially significantly superior to the traditional multiplicative update rule algorithm in terms of convergence and clustering performance.

In short, the main contribution of this paper is to propose a new, convergent BNMtF algorithm, which solves the shortcomings of existing algorithms in terms of convergence and shows better clustering effects in practical applications.