Abstract:Multivariate time-series data in numerous real-world applications (e.g., healthcare and industry) are informative but challenging due to the lack of labels and high dimensionality. Recent studies in self-supervised learning have shown their potential in learning rich representations without relying on labels, yet they fall short in learning disentangled embeddings and addressing issues of inductive bias (e.g., transformation-invariance). To tackle these challenges, we propose TimeDRL, a generic multivariate time-series representation learning framework with disentangled dual-level embeddings. TimeDRL is characterized by three novel features: (i) disentangled derivation of timestamp-level and instance-level embeddings from patched time-series data using a [CLS] token strategy; (ii) utilization of timestamp-predictive and instance-contrastive tasks for disentangled representation learning, with the former optimizing timestamp-level embeddings with predictive loss, and the latter optimizing instance-level embeddings with contrastive loss; and (iii) avoidance of augmentation methods to eliminate inductive biases, such as transformation-invariance from cropping and masking. Comprehensive experiments on 6 time-series forecasting datasets and 5 time-series classification datasets have shown that TimeDRL consistently surpasses existing representation learning approaches, achieving an average improvement of forecasting by 58.02% in MSE and classification by 1.48% in accuracy. Furthermore, extensive ablation studies confirmed the relative contribution of each component in TimeDRL's architecture, and semi-supervised learning evaluations demonstrated its effectiveness in real-world scenarios, even with limited labeled data. The code is available at <a class="link-external link-https" href="https://github.com/blacksnail789521/TimeDRL" rel="external noopener nofollow">this https URL</a>.

A Deep Multi-Task Representation Learning Method for Time Series Classification and Retrieval.

Multilevel Dynamic Time Warping: A Parameter-Light Method for Fast Time Series Classification

Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series

Multitask Learning for Time Series Data with 2D Convolution

Multi-scale Self-Supervised Representation Learning with Temporal Alignment for Multi-Rate Time Series Modeling

Multi-Task Self-Supervised Time-Series Representation Learning

TimeDRL: Disentangled Representation Learning for Multivariate Time-Series

FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification

Exploiting Multi-Channels Deep Convolutional Neural Networks for Multivariate Time Series Classification

Multi-Faceted Representation Learning with Hybrid Architecture for Time Series Classification

Deep R-Th Root of Rank Supervised Joint Binary Embedding for Multivariate Time Series Retrieval

Time-frequency deep metric learning for multivariate time series classification

Self-Supervised Learning of Disentangled Representations for Multivariate Time-Series

TV-Net: Temporal-Variable feature harmonizing Network for multivariate time series classification and interpretation

Multi-task Representation Learning for Demographic Prediction.

MRM2: Multi-Relationship Modeling Module for Multivariate Time Series Classification.

Densely Knowledge-Aware Network for Multivariate Time Series Classification

Time Series Representation Models

Adaptive Transfer Learning of Multi-View Time Series Classification

Time-Series Representation Learning via Temporal and Contextual Contrasting

MDLR: A Multi-Task Disentangled Learning Representations for unsupervised time series domain adaptation