Abstract:Traditional Recurrent Neural Networks assume vectorized data as inputs. However many data from modern science and technology come in certain structures such as tensorial time series data. To apply the recurrent neural networks for this type of data, a vectorisation process is necessary, while such a vectorisation leads to the loss of the precise information of the spatial or longitudinal dimensions. In addition, such a vectorized data is not an optimum solution for learning the representation of the longitudinal data. In this paper, we propose a new variant of tensorial neural networks which directly take tensorial time series data as inputs. We call this new variant as Tensorial Recurrent Neural Network (TRNN). The proposed TRNN is based on tensor Tucker decomposition.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the limitations of traditional Recurrent Neural Networks (RNNs) when dealing with high - order tensor time - series data. Specifically, traditional RNNs assume that the input data is in vectorized form, but much data in modern science and technology has a specific structure, such as tensor time - series data. In order to apply these data to RNNs, vectorization processing is usually required, which will lead to the loss of accurate information in the spatial or longitudinal dimension. Moreover, this vectorized data is not the optimal solution for learning the representation of longitudinal data. To address this problem, the paper proposes a new Tensorial Recurrent Neural Network (TRNN), which can directly take tensor time - series data as input. Based on the tensor Tucker decomposition, TRNN aims to preserve the spatial structure information of the data and improve the learning efficiency and accuracy of multi - dimensional time - series data. The main contributions of the paper include: 1. **Proposing the TRNN model**: It directly processes tensor time - series data, avoiding the information loss caused by the vectorization process. 2. **Designing tensor LSTM (tLSTM) and tensor GRU (tGRU)**: It extends the traditional LSTM and GRU models, enabling them to process tensor data. 3. **Deriving the back - propagation algorithm**: It elaborately derives the back - propagation algorithms of tLSTM and tGRU to ensure the effective training of the model. 4. **Experimental verification**: It verifies the performance of the TRNN model in processing multi - dimensional time - series data through actual data sets, showing its advantages over traditional RNNs. Through these contributions, the paper aims to provide a more effective tool for processing and analyzing multi - dimensional time - series data with complex structures.

Tensorial Recurrent Neural Networks for Longitudinal Data Analysis

RSNN: Recurrent Spiking Neural Networks for Dynamic Spatial-Temporal Information Processing

Network of Tensor Time Series

Tensor-Train Recurrent Neural Networks for Interpretable Multi-Way Financial Forecasting

An Efficient Approach to Regression Problems with Tensor Neural Networks

Gated Recurrent Neural Tensor Network

An Improved Time Feedforward Connections Recurrent Neural Networks

TTH-RNN: Tensor-Train Hierarchical Recurrent Neural Network for Video Summarization

TR-TransGAN: Temporal Recurrent Transformer Generative Adversarial Network for Longitudinal MRI Dataset Expansion

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series Tasks

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics

Factor Augmented Tensor-on-Tensor Neural Networks

Tensor Networks for Latent Variable Analysis. Part I: Algorithms for Tensor Train Decomposition

Long-Short Term Spatiotemporal Tensor Prediction for Passenger Flow Profile

Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition

Tensor train decompositions on recurrent networks

Temporal Reversed Training for Spiking Neural Networks with Generalized Spatio-Temporal Representation

SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecasting

Compressing Recurrent Neural Network with Tensor Train

Tensor Generalized Estimating Equations for Longitudinal Imaging Analysis