Abstract:Recently, many works have demonstrated that Symmetric Non-negative Matrix Factorization~(SymNMF) enjoys a great superiority for various clustering tasks. Although the state-of-the-art algorithms for SymNMF perform well on synthetic data, they cannot consistently obtain satisfactory results with desirable properties and may fail on real-world tasks like clustering. Considering the flexibility and strong representation ability of the neural network, in this paper, we propose a neural network called SymNMF-Net for the Symmetric NMF problem to overcome the shortcomings of traditional optimization algorithms. Each block of SymNMF-Net is a differentiable architecture with an inversion layer, a linear layer and ReLU, which are inspired by a traditional update scheme for SymNMF. We show that the inference of each block corresponds to a single iteration of the optimization. Furthermore, we analyze the constraints of the inversion layer to ensure the output stability of the network to a certain extent. Empirical results on real-world datasets demonstrate the superiority of our SymNMF-Net and confirm the sufficiency of our theoretical analysis.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the performance problem of Symmetric Non - negative Matrix Factorization (SymNMF) in clustering tasks. Specifically: 1. **Limitations of existing algorithms**: - Although the existing SymNMF algorithms perform well on synthetic data, in practical tasks such as clustering, they cannot always obtain satisfactory results. - Existing algorithms are prone to getting trapped in local optimal solutions and are sensitive to initial values, making it difficult to ensure the stability and consistency of the results. 2. **Proposed new method**: - The paper proposes a neural - network - based method - SymNMF - Net to solve the SymNMF problem. - Each SymNMF - Net block contains an invertible layer, a linear layer and a ReLU layer, and the design of these layers is inspired by the traditional SymNMF update scheme. 3. **Improved objectives**: - Improve the performance of SymNMF in practical tasks (such as clustering) and ensure that it can converge more stably to better solutions. - Overcome the shortcomings of traditional optimization algorithms by introducing the flexibility and strong representational ability of neural networks, so as to achieve better results on real - world datasets. ### Formula summary - **Formula for the classical SymNMF problem**: \[ \min_{U \in \mathbb{R}^{n \times r}} \frac{1}{2}\|X - UU^{\top}\|_F^2, \quad \text{s.t. } U \geq 0 \] - **Formula for the SymNMF problem with auxiliary variables**: \[ \min_{U, V \in \mathbb{R}^{n \times r}} \frac{1}{2}\|X - UV^{\top}\|_F^2+\frac{\lambda}{2}\|U - V\|_F^2, \quad \text{s.t. } U \geq 0, V \geq 0 \] - **Forward propagation formula for SymNMF - Net block**: \[ U_i=\max\left\{(X + \lambda I_n)U_{i - 1}(U_{i - 1}^{\top}U_{i - 1}+\lambda I_r)^{-1}, 0\right\} \] ### Conclusion In view of some limitations of the traditional optimization algorithms for the symmetric non - negative matrix factorization problem, the paper proposes the SymNMF - Net method based on neural networks. This method not only improves the performance of SymNMF in practical tasks, but also ensures more stable convergence and better results by introducing the flexibility and strong representational ability of neural networks. The experimental results show that SymNMF - Net outperforms the existing methods in both image and text clustering tasks.

SymNMF-Net for The Symmetric NMF Problem

A Two-Phase Algorithm for Robust Symmetric Non-Negative Matrix Factorization

Self-supervised Symmetric Nonnegative Matrix Factorization

Learnable Similarity and Dissimilarity Guided Symmetric Non-Negative Matrix Factorization

Block-Diagonal Guided Symmetric Nonnegative Matrix Factorization

Semi-supervised Symmetric Non-negative Matrix Factorization with Low-Rank Tensor Representation

A Provable Splitting Approach for Symmetric Nonnegative Matrix Factorization

Symmetric-threshold ReLU for Fast and Nearly Lossless ANN-SNN Conversion

Semisupervised Adaptive Symmetric Non-Negative Matrix Factorization.

WSNMF: Weighted Symmetric Nonnegative Matrix Factorization for attributed graph clustering

Randomized Algorithms for Symmetric Nonnegative Matrix Factorization

Adaptive graph regularized nonnegative matrix factorization for data representation

System partitioning on MCM using a new neural network model

Multi-constrained Symmetric Nonnegative Latent Factor Analysis for Accurately Representing Large-scale Undirected Weighted Networks

Optimization and expansion of non-negative matrix factorization

Adaptive computation of the Symmetric Nonnegative Matrix Factorization (NMF)

Modern synergetic neural network for imbalanced small data classification

Monomial Matrix Group Equivariant Neural Functional Networks

NeNMF: an Optimal Gradient Method for Nonnegative Matrix Factorization.

An Alternating-Direction-Method of Multipliers-Incorporated Approach to Symmetric Non-Negative Latent Factor Analysis

Symmetry-constrained Non-negative Matrix Factorization Approach for Highly-Accurate Community Detection