Abstract:Revisiting PCA for Time Series Reduction in Temporal Dimension; Jiaxin Gao, Wenbo Hu, Yuntian Chen; Deep learning has significantly advanced time series analysis (TSA), enabling the extraction of complex patterns for tasks like classification, forecasting, and regression. Although dimensionality reduction has traditionally focused on the variable space-achieving notable success in minimizing data redundancy and computational complexity-less attention has been paid to reducing the temporal dimension. In this study, we revisit Principal Component Analysis (PCA), a classical dimensionality reduction technique, to explore its utility in temporal dimension reduction for time series data. It is generally thought that applying PCA to the temporal dimension would disrupt temporal dependencies, leading to limited exploration in this area. However, our theoretical analysis and extensive experiments demonstrate that applying PCA to sliding series windows not only maintains model performance, but also enhances computational efficiency. In auto-regressive forecasting, the temporal structure is partially preserved through windowing, and PCA is applied within these windows to denoise the time series while retaining their statistical information. By preprocessing time-series data with PCA, we reduce the temporal dimensionality before feeding it into TSA models such as Linear, Transformer, CNN, and RNN architectures. This approach accelerates training and inference and reduces resource consumption. Notably, PCA improves Informer training and inference speed by up to 40% and decreases GPU memory usage of TimesNet by 30%, without sacrificing model accuracy. Comparative analysis against other reduction methods further highlights the effectiveness of PCA in improving the efficiency of TSA models.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in time - series analysis (TSA), how to effectively reduce redundant information in the time dimension to relieve the computational burden and improve model efficiency without sacrificing model performance. Specifically, traditional dimension - reduction methods mainly focus on dimension - reduction in the variable space, while there is less research on dimension - reduction in the time dimension. The author re - examines principal component analysis (PCA), a classic dimension - reduction technique, and explores its potential application in dimension - reduction in the time dimension of time - series data. ### Main problems and challenges 1. **Long time - series length**: Time - series data usually has a long time span, resulting in high computational complexity and large memory consumption when processing these data. 2. **Redundant information**: There is a large amount of redundant information in the original time - series data. Directly inputting these data will increase the difficulty of model training and inference. 3. **Maintaining time - dependence**: In time - series data, the order and correlation between time steps are very important. Therefore, the dimension - reduction method needs to maintain these characteristics as much as possible while reducing the dimension. ### Solutions The author proposes to apply PCA to time - series data in a sliding window to solve the problem in the following ways: - **Maintaining model performance**: By applying PCA within the sliding window, the statistical information of the time - series can be maintained while reducing the dimension, so that the prediction performance of the model is not impaired. - **Improving computational efficiency**: PCA can effectively remove noise and redundant information, reduce the correlation between different time steps, thereby reducing the risk of over - fitting and increasing the training and inference speed. - **Applicable to multiple models**: PCA pre - processing can be applied to multiple deep - learning models such as linear models, Transformer, CNN and RNN, which widely improves the efficiency of these models. ### Experimental results By conducting experiments on multiple typical time - series tasks (such as classification, prediction and exogenous regression), the author proves the effectiveness of PCA pre - processing. For example: - PCA pre - processing improves the training and inference speed of Informer by 40% and reduces the GPU memory usage of TimesNet by 30%. - In the time - series classification task, PCA pre - processing not only retains the main information of the time - series, but also accelerates the training and inference process. In conclusion, this paper provides an effective dimension - reduction method for time - series analysis by re - examining the application of PCA in the time dimension, which reduces the computational burden and maintains the model performance at the same time.

Revisiting PCA for time series reduction in temporal dimension

Temporal Streaming Batch Principal Component Analysis for Time Series Classification

DROP: Dimensionality Reduction Optimization for Time Series

Deep Residual Principal Component Analysis As Feature Engineering for Industrial Data Analytics

Supervised Linear Dimension-Reduction Methods: Review, Extensions, and Comparisons

Self-paced Principal Component Analysis

nPCA: a linear dimensionality reduction method using a multilayer perceptron

TemporalPaD: a reinforcement-learning framework for temporal feature representation and dimension reduction

GraphPCA: a fast and interpretable dimension reduction algorithm for spatial transcriptomics data

Improved Dimensionality Reduction of various Datasets using Novel Multiplicative Factoring Principal Component Analysis (MPCA)

Supervised Discriminative Sparse PCA with Adaptive Neighbors for Dimensionality Reduction

Overview of PCA-Based Statistical Process-Monitoring Methods for Time-Dependent, High-Dimensional Data

PCA-KL: a parametric dimensionality reduction approach for unsupervised metric learning

Principal component analysis for second-order stationary vector time series

Comparing dimensionality reduction techniques for visual analysis of the LSTM hidden activity on multi-dimensional time series modeling

Asymptotic theory of principal component analysis for time series data with cautionary comments

Adaptive dimensionality reduction for neural network-based online principal component analysis

An Exploration of the Application of Principal Component Analysis in Big Data Processing

Comparative Analysis of 2D-PCA Based Dimensionality Reduction and Feature Extraction

Dynamic Principal Subspaces in High Dimensions

Nonlinear Dimensionality Reduction for Discriminative Analytics of Multiple Datasets