Abstract:Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi--modal datasets, which are often conveniently represented as multiway arrays or tensors. It is therefore timely and valuable for the multidisciplinary research community to review tensor decompositions and tensor networks as emerging tools for large-scale data analysis and data mining. We provide the mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations. Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker models, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, multiway component analysis, multilinear blind source separation, tensor completion, linear/multilinear dimensionality reduction, large-scale optimization problems, symmetric eigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations, pseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis (CCA) (This is Part 1)

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is how to overcome the "curse of dimensionality" when dealing with large - scale, multi - dimensional data, and to provide an efficient and scalable method to solve large - scale optimization problems. Specifically, the authors focus on how to perform dimension reduction and optimization through low - rank tensor networks, so as to be able to handle very large data sets. ### Main Problem Analysis 1. **Curse of Dimensionality**: - The curse of dimensionality refers to the problem that as the data dimension increases, the required storage space and computing resources grow exponentially. For example, the number of elements of an \(N\)-th order tensor is \(I^N\), where \(I\) is the size of each mode. When \(N\) is large, this growth makes data processing extremely difficult. - The paper proposes to alleviate this problem through low - rank tensor network representations. These methods can compress high - order tensors into combinations of low - order core tensors, thereby significantly reducing storage requirements and computational complexity. 2. **Large - Scale Optimization Problems**: - When dealing with large - scale data, traditional optimization algorithms (such as convex optimization algorithms) are often unable to effectively scale to large data sets. Therefore, new methods that can maintain linear or sub - linear complexity as the data scale increases need to be developed. - The authors explore how to use low - rank tensor network decompositions (such as Tucker decomposition and Tensor Train (TT) decomposition) to solve such problems. These methods can not only compress data, but also transform the originally intractable large - scale optimization problems into a series of smaller, easier - to - solve sub - problems. ### Solutions - **Tensor Network Decomposition**: The paper focuses on several low - rank tensor network decomposition methods, including CP decomposition, Tucker decomposition, and Tensor Train (TT) decomposition, etc. These methods achieve effective data compression by decomposing high - order tensors into products of multiple low - order tensors. - **Distributed and Parallel Computing**: To further improve efficiency, the paper also discusses how to use distributed and parallel computing techniques to handle large - scale data. By dividing the data into multiple small pieces and processing them in parallel on different computing nodes, the computing process can be significantly accelerated. - **Application Examples**: The paper demonstrates the effectiveness of these methods in various practical applications, including large - scale optimization problems in multimodal data analysis, image processing, machine learning, and other fields. In conclusion, this paper aims to provide an effective solution to deal with large - scale, multi - dimensional data processing and optimization problems through low - rank tensor network decomposition methods, thereby promoting the development of the big data analysis and machine learning fields.

Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems

Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions

Tensor Denoising Using Low-Rank Tensor Train Decomposition

Tensor Methods in High Dimensional Data Analysis: Opportunities and Challenges

Decomposition of Big Tensors With Low Multilinear Rank

Tensor Networks for Latent Variable Analysis. Part I: Algorithms for Tensor Train Decomposition

Tensor Decomposition for Signal Processing and Machine Learning

Introduction to Tensor Decompositions and their Applications in Machine Learning

Tensor decompositions: computations, applications, and challenges

An Iterative Reweighted Method for Tucker Decomposition of Incomplete Multiway Tensors

Tensor Decompositions: A New Concept in Brain Data Analysis?

Distributed Non-Negative Tensor Train Decomposition

A Multi-resolution Low-rank Tensor Decomposition

A Tensor-Based Approach for Big Data Representation and Dimensionality Reduction

Multilinear Low-Rank Tensors on Graphs & Applications

Support Vector Machine Based On Low-Rank Tensor Train Decomposition For Big Data Applications

An Iterative Reweighted Method for Tucker Decomposition of Incomplete Tensors

Extension of PCA to Higher Order Data Structures: An Introduction to Tensors, Tensor Decompositions, and Tensor PCA

DeepTensor: Low-Rank Tensor Decomposition with Deep Network Priors