Abstract:The tensor train decomposition decomposes a tensor into a "train" of 3-way tensors that are interconnected through the summation of auxiliary indices. The decomposition is stable, has a well-defined notion of rank and enables the user to perform various linear algebra operations on vectors and matrices of exponential size in a computationally efficient manner. The tensor ring decomposition replaces the train by a ring through the introduction of one additional auxiliary variable. This article discusses a major issue with the tensor ring decomposition: its inability to compute an exact minimal-rank decomposition from a decomposition with sub-optimal ranks. Both the contraction operation and Hadamard product are motivated from applications and it is shown through simple examples how the tensor ring-rounding procedure fails to retrieve minimal-rank decompositions with these operations. These observations, together with the already known issue of not being able to find a best low-rank tensor ring approximation to a given tensor indicate that the applicability of tensor rings is severely limited.
What problem does this paper attempt to address?
This paper aims to explore a major problem in Tensor Ring Decomposition (TR): it is impossible to calculate the exact minimum - rank decomposition from the decomposition of sub - optimal rank. Specifically, the paper shows through simple examples that during the contraction operation and Hadamard product, the rounding process of the tensor ring cannot recover the minimum - rank decomposition. These problems, combined with the known problem of being unable to find the best low - rank tensor ring approximation for a given tensor, indicate that the application range of the tensor ring is severely limited.
### Main problems
1. **Failure to recover the minimum - rank decomposition**:
- The paper points out that after certain linear algebra operations (such as matrix multiplication and Hadamard product), the rank of the tensor ring may increase. Although in theory, these increased ranks can be reduced by rounding algorithms, experimental results show that for tensor ring decomposition, this process often fails, that is, it is impossible to recover to the minimum rank.
2. **Contraction operation and Hadamard product**:
- The contraction operation and Hadamard product are common operations in practical applications. The paper shows through specific numerical experiments that after these operations, the rounding algorithm of the tensor ring cannot effectively recover the minimum - rank decomposition.
3. **Comparison with Tensor Train (TT)**:
- In contrast, after similar operations, the Tensor Train decomposition can be successfully rounded back to the minimum rank by the rounding algorithm. This further highlights the limitations of tensor ring decomposition.
### Experimental results
- **Matrix multiplication**:
- For a rectangular matrix \(A\) of size \(I\times J^{d}\), if \(A\) has a tensor ring representation with a uniform rank of \(R\), then the tensor ring rank of \(A A^{T}\) will become \(R^{2}\). Experimental results show that even using a high - precision rounding algorithm, it is impossible to round \(R^{2}\) to the minimum rank of 1.
- **Hadamard product**:
- For a 6 - dimensional cube tensor \(A\) with each tensor ring rank of \(R\), experimental results show that the tensor ring rank of the Hadamard product of \(A\) with itself will become \(R^{2}\). Similarly, the rounding algorithm cannot round \(R^{2}\) to a smaller rank.
### Conclusion
Through experimental and theoretical analysis, the paper reveals the problem that tensor ring decomposition cannot effectively recover the minimum rank after common linear algebra operations. This finding poses a challenge to the practical application of tensor ring decomposition, especially in scenarios that require multiple iterations or complex operations. In contrast, the Tensor Train decomposition is more stable and reliable in this regard. Therefore, the paper suggests that when choosing a tensor decomposition method, the limitations of tensor ring decomposition should be carefully considered.