Randomized coupled decompositions

Erna Begovic,Anita Carevic,Ivana Sain Glibic

2024-11-04

Abstract:Coupled decompositions are a widely used tool for data fusion. This paper studies the coupled matrix factorization (CMF) where two matrices $X$ and $Y$ are represented in a low-rank format sharing one common factor, as well as the coupled matrix and tensor factorization (CMTF) where a matrix $Y$ and a tensor $\mathcal{X}$ are represented in a low-rank format sharing a factor matrix. We show that these problems are equivalent to the low-rank approximation of the matrix $[X \ Y]$ for CMF, that is $[X_{(1)} \ Y]$ for CMTF. Then, in order to speed up computation process, we adapt several randomization techniques, namely, randomized SVD, randomized subspace iteration, and randomized block Krylov iteration to the algorithms for coupled decompositions. We present extensive results of the numerical tests. Furthermore, as a novel approach and with a high success rate, we apply our randomized algorithms to the face recognition problem.

Numerical Analysis

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to accelerate Coupled Matrix Factorization (CMF) and Coupled Matrix and Tensor Factorization (CMTF) through randomization techniques to improve computational efficiency while maintaining high precision. Specifically:

1. **Coupled Matrix Factorization (CMF)**:
- The paper studies the low - rank representation problems of two matrices \(X\in\mathbb{R}^{m\times n_1}\) and \(Y\in\mathbb{R}^{m\times n_2}\), where these two matrices share a common factor.
- The goal is to minimize the objective function:
\[
f(U, V, W)=\|X - UV^T\|_F^2+\|Y - UW^T\|_F^2\rightarrow\min
\]
- The author proves that this problem is equivalent to performing a low - rank approximation on the matrix \([X\ Y]\in\mathbb{R}^{m\times(n_1 + n_2)}\).

2. **Coupled Matrix and Tensor Factorization (CMTF)**:
- It studies the low - rank representation problems of a matrix \(Y\in\mathbb{R}^{m\times n}\) and a tensor \(X\in\mathbb{R}^{m\times n_2\times n_3}\), which also share a factor matrix.
- The author simplifies the CMTF problem to performing a low - rank approximation on the matrix \([X(1)\ Y]\in\mathbb{R}^{m\times(n_2n_3 + n)}\), where \(X(1)\) is the first - mode unfolding of the tensor \(X\).

In order to accelerate these decomposition processes, the author introduces several randomization techniques, including:
- Randomized Singular Value Decomposition (Randomized SVD)
- Randomized Subspace Iteration
- Randomized Block Krylov Iteration

These randomized methods can significantly reduce the computation time, especially when dealing with large - scale data sets. In addition, the author also applies these randomized algorithms to the face recognition problem and has achieved a high success rate.