Abstract:Multivariate time-series data in numerous real-world applications (e.g., healthcare and industry) are informative but challenging due to the lack of labels and high dimensionality. Recent studies in self-supervised learning have shown their potential in learning rich representations without relying on labels, yet they fall short in learning disentangled embeddings and addressing issues of inductive bias (e.g., transformation-invariance). To tackle these challenges, we propose TimeDRL, a generic multivariate time-series representation learning framework with disentangled dual-level embeddings. TimeDRL is characterized by three novel features: (i) disentangled derivation of timestamp-level and instance-level embeddings from patched time-series data using a [CLS] token strategy; (ii) utilization of timestamp-predictive and instance-contrastive tasks for disentangled representation learning, with the former optimizing timestamp-level embeddings with predictive loss, and the latter optimizing instance-level embeddings with contrastive loss; and (iii) avoidance of augmentation methods to eliminate inductive biases, such as transformation-invariance from cropping and masking. Comprehensive experiments on 6 time-series forecasting datasets and 5 time-series classification datasets have shown that TimeDRL consistently surpasses existing representation learning approaches, achieving an average improvement of forecasting by 58.02% in MSE and classification by 1.48% in accuracy. Furthermore, extensive ablation studies confirmed the relative contribution of each component in TimeDRL's architecture, and semi-supervised learning evaluations demonstrated its effectiveness in real-world scenarios, even with limited labeled data. The code is available at <a class="link-external link-https" href="https://github.com/blacksnail789521/TimeDRL" rel="external noopener nofollow">this https URL</a>.

A Novel Embedded Discretization-Based Deep Learning Architecture for Multivariate Time Series Classification

Multivariate Time Series Classification: A Deep Learning Approach

Exploiting Multi-Channels Deep Convolutional Neural Networks for Multivariate Time Series Classification

A Novel Encoder-Decoder Model for Multivariate Time Series Forecasting

Densely Knowledge-Aware Network for Multivariate Time Series Classification

DTW-Merge: A Novel Data Augmentation Technique for Time Series Classification

An Enhancing Timeseries Anomaly Detection Using LSTM and Bi-LSTM Architectures

A deep graph kernel-based time series classification algorithm

Multivariate Time Series Early Classification Across Channel and Time Dimensions

A Review of Deep Learning Models for Time Series Prediction

Deep learning for time series classification: a review

Deep Learning for Time-Series Analysis

Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey

MTS2Graph: Interpretable Multivariate Time Series Classification with Temporal Evolving Graphs

TimeDRL: Disentangled Representation Learning for Multivariate Time-Series

Time series forecasting and anomaly detection using deep learning

Disentangling Domain and General Representations for Time Series Classification

Machine Learning of Time Series Using Time-Delay Embedding and Precision Annealing

View-Adaptive Weighted Deep Transfer Learning for Distributed Time-Series Classification