Abstract:Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenges in high - dimensional data processing and large - scale optimization, especially achieving efficient data compression and processing through Tensor Networks technology. Specifically, the paper focuses on how to use tensor network models for the super - compressed representation of data/parameters and the optimization of their related cost functions, aiming to overcome or eliminate the curse of dimensionality, that is, the problem that the computational complexity increases exponentially as the data dimension increases. ### Main research directions 1. **Tensor network models**: The paper mainly introduces tensor network models, especially Tensor Train (TT) and Hierarchical Tucker (HT) decompositions. These models can approximately represent high - order tensors with a lower rank, thus achieving efficient data compression. 2. **Distributed computing**: Through low - rank tensor approximation and complex contractions of core tensors, tensor networks can process large - scale data sets in a distributed computing environment, thereby reducing or eliminating the impact of the curse of dimensionality. 3. **Application fields**: The paper explores the applications of tensor networks in machine learning and data analysis, including generalized regression and classification (support tensor machines, canonical correlation analysis, high - order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and the optimization of deep neural networks. ### Specific problems - **Data compression and representation**: How to effectively compress and represent high - dimensional data through tensor network technology for subsequent multi - way data analysis. - **Large - scale optimization**: How to use tensor network models to improve computational efficiency in large - scale optimization problems, especially when dealing with large - scale data sets. - **Curse of dimensionality**: How to overcome the curse of dimensionality through tensor network technology so that the feasibility and effectiveness of computation can still be maintained in high - dimensional data processing. ### Methods and techniques - **Tensorization**: Convert low - order data formats (such as vectors, matrices) into high - order tensors for efficient processing using tensor networks. - **Low - rank approximation**: Compress data through low - rank tensor approximation (such as TT and HT decompositions) to reduce storage and computational requirements. - **Distributed computing**: Utilize the distributed computing capabilities of tensor networks to process large - scale data sets. ### Application examples - **Signal processing**: In Blind Source Separation (BSS), through tensorization and low - rank approximation, the original signals can be separated from the mixed signals. - **Image processing**: In multi - channel EEG signal processing, through tensorization and low - rank approximation, useful information can be extracted. - **Machine learning**: In the optimization of deep neural networks, through tensor network technology, the training efficiency and performance of the model can be improved. In conclusion, through in - depth exploration of the applications of tensor network technology in high - dimensional data processing and large - scale optimization, this paper aims to provide an effective solution to meet the challenges in modern data science.

Tensor Networks for Dimensionality Reduction and Large-Scale Optimizations. Part 2 Applications and Future Perspectives

Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1

Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems

Tensor Denoising Using Low-Rank Tensor Train Decomposition

Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions

Tensor Networks for Latent Variable Analysis. Part I: Algorithms for Tensor Train Decomposition

Introduction to Tensor Decompositions and their Applications in Machine Learning

Tensor decompositions: computations, applications, and challenges

Distributed Non-Negative Tensor Train Decomposition

Tensor Decomposition for Signal Processing and Machine Learning

Tensor Methods in High Dimensional Data Analysis: Opportunities and Challenges

Tensor decompositions and algorithms, with applications to tensor learning

Decomposition of Big Tensors With Low Multilinear Rank

On Optimizing Distributed Tucker Decomposition for Dense Tensors

Incremental Tensor Analysis

Reducing Computational Complexity of Tensor Contractions via Tensor-Train Networks

Tensor Decompositions: A New Concept in Brain Data Analysis?

A Tensor-Based Approach for Big Data Representation and Dimensionality Reduction

Scalable Symmetric Tucker Tensor Decomposition

Fully-Connected Tensor Network Decomposition and Its Application to Higher-Order Tensor Completion.

Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis