Abstract:The era of exascale computing opens new venues for innovations and discoveries in many scientific, engineering, and commercial fields. However, with the exaflops also come the extra-large high-dimensional data generated by high-performance computing. High-dimensional data is presented as multidimensional arrays, aka tensors. The presence of latent (not directly observable) structures in the tensor allows a unique representation and compression of the data by classical tensor factorization techniques. However, the classical tensor methods are not always stable or they can be exponential in their memory requirements, which makes them not suitable for high-dimensional tensors. Tensor train (TT) is a state-of-the-art tensor network introduced for factorization of high-dimensional tensors. TT transforms the initial high-dimensional tensor in a network of three-dimensional tensors that requires only a linear storage. Many real-world data, such as, density, temperature, population, probability, etc., are non-negative and for an easy interpretation, the algorithms preserving non-negativity are preferred. Here, we introduce a distributed non-negative tensor-train and demonstrate its scalability and the compression on synthetic and real-world big datasets.

What problem does this paper attempt to address?

This paper aims to solve the storage and processing challenges of high - dimensional data in high - performance computing. Specifically, the paper attempts to solve the following core problems: 1. **Compression of high - dimensional data**: With the development of high - performance computing (HPC), the amount of generated data is increasingly large and has extremely high dimensions. These high - dimensional data are usually represented in the form of multi - dimensional arrays (i.e., tensors). Traditional compression techniques often have poor performance when dealing with such data because they require high precision and prediction accuracy. Therefore, the paper proposes a new method to effectively compress high - dimensional data while maintaining the non - negativity of the data for easy interpretation and application. 2. **Non - negativity preservation**: Many real - world data, such as density, temperature, population, probability, etc., are non - negative. In order to maintain the non - negativity of these data and thus retain their interpretability and significance, the paper introduces the Non - Negative Tensor Train (nTT) decomposition method. This method can not only effectively compress data but also ensure that the low - dimensional factors after decomposition are non - negative. 3. **Distributed computing**: Since the processing of large - scale high - dimensional data requires a large amount of computing resources, traditional single - machine algorithms cannot meet this demand. The paper proposes a distributed non - negative tensor chain algorithm to improve the scalability and efficiency of the algorithm through parallel computing. This enables the method to be applied to actual large - scale data sets. ### Main contributions of the paper 1. **Distributed non - negative tensor chain algorithm**: The paper proposes a distributed non - negative tensor chain (nTT) algorithm, which is based on distributed non - negative matrix factorization (NMF) and tensor expansion and reshaping operations. Through distributed computing, this algorithm can achieve efficient compression and decomposition on large - scale data sets. 2. **Algorithm performance analysis**: The paper analyzes in detail the performance of the proposed distributed nTT algorithm, including its compression ability and scalability on synthetic data and real data sets. The experimental results show that the algorithm performs well on processors and data sets of different scales. 3. **Application examples**: The paper also shows the application of this algorithm on actual data sets, such as Extended Yale Face Dataset B. The experimental results show that the algorithm can effectively compress facial image data while maintaining the non - negativity and interpretability of the data. ### Formula representation - **Tensor chain decomposition**: \[ A = G^{(1)} \circ G^{(2)} \circ \ldots \circ G^{(d)} \] where \( A \) is a \( d \)-dimensional tensor, \( G^{(i)} \in \mathbb{R}^{r_{i - 1} \times n_i \times r_i}\) is the core of the tensor chain, and \( r_0 = r_d = 1\). - **Non - negative tensor chain decomposition**: \[ A=\sum_{k_1, \ldots, k_{d - 1}} G^{(1)}_{i_1, k_1} \cdot G^{(2)}_{k_1, i_2, k_2} \cdot \ldots \cdot G^{(d)}_{k_{d - 1}, i_d} \] - **NMF decomposition**: \[ X \approx WH \] where \( X \in \mathbb{R}^{m \times n}\), \( W \in \mathbb{R}^{m \times r}\), \( H \in \mathbb{R}^{r \times n}\). ### Summary This paper solves the storage and processing problems of high - dimensional data in high - performance computing by proposing a distributed non - negative tensor chain algorithm. This algorithm can not only effectively compress data but also maintain the non - negativity of data, and is suitable for processing large - scale data sets. Through detailed performance analysis and practical application examples, the paper verifies the effectiveness and practicality of this method.

Distributed Non-Negative Tensor Train Decomposition

Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems

A Distributed Tensor-Train Decomposition Method for Cyber-Physical-Social Services.

Tensor Networks for Latent Variable Analysis. Part I: Algorithms for Tensor Train Decomposition

Tensor Train Decomposition for Solving Large-Scale Linear Equations.

Hardware-Enabled Efficient Data Processing with Tensor-Train Decomposition

Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1

Nonlinear Tensor Train Format for Deep Neural Network Compression

High-Performance Tensor-Train Primitives Using GPU Tensor Cores

Parallel algorithms for computing the tensor-train decomposition

Reducing Computational Complexity of Tensor Contractions via Tensor-Train Networks

Computational and Statistical Guarantees for Tensor-on-Tensor Regression with Tensor Train Decomposition

Distributed Flexible Nonlinear Tensor Factorization

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Decomposition of Big Tensors With Low Multilinear Rank

Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions

Scalable CP Decomposition for Tensor Learning using GPU Tensor Cores

Tensor-Train Compression of Discrete Element Method Simulation Data

Compressing 3DCNNs based on tensor train decomposition

Nonnegative Canonical Tensor Decomposition with Linear Constraints: nnCANDELINC

Hybrid Tensor Decomposition in Neural Network Compression