Abstract:Contrastive learning underpins most current self-supervised time series representation methods. The strategy for constructing positive and negative sample pairs significantly affects the final representation quality. However, due to the continuous nature of time series semantics, the modeling approach of contrastive learning struggles to accommodate the characteristics of time series data. This results in issues such as difficulties in constructing hard negative samples and the potential introduction of inappropriate biases during positive sample construction. Although some recent works have developed several scientific strategies for constructing positive and negative sample pairs with improved effectiveness, they remain constrained by the contrastive learning framework. To fundamentally overcome the limitations of contrastive learning, this paper introduces Frequency-masked Embedding Inference (FEI), a novel non-contrastive method that completely eliminates the need for positive and negative samples. The proposed FEI constructs 2 inference branches based on a prompting strategy: 1) Using frequency masking as prompts to infer the embedding representation of the target series with missing frequency bands in the embedding space, and 2) Using the target series as prompts to infer its frequency masking embedding. In this way, FEI enables continuous semantic relationship modeling for time series. Experiments on 8 widely used time series datasets for classification and regression tasks, using linear evaluation and end-to-end fine-tuning, show that FEI significantly outperforms existing contrastive-based methods in terms of generalization. This study provides new insights into self-supervised representation learning for time series. The code is available at <a class="link-external link-https" href="https://github.com/USTBInnovationPark/Frequency-masked-Embedding-Inference" rel="external noopener nofollow">this https URL</a>.

Time-Series Representation Feature Refinement with a Learnable Masking Augmentation Framework in Contrastive Learning

Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series

Towards Unsupervised Time Series Representation Learning: A Decomposition Perspective

DABaCLT: A Data Augmentation Bias-Aware Contrastive Learning Framework for Time Series Representation

Self-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification

Time-Series Representation Learning Via Dual Reference Contrasting

Frequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning

Time-series representation learning via Time-Frequency Fusion Contrasting

Time-Series Representation Learning via Temporal and Contextual Contrasting

Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Series

Multi-Task Self-Supervised Time-Series Representation Learning

MHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time Series

Time Series Contrastive Learning with Information-Aware Augmentations

Time-Series Forecasting Through Contrastive Learning with a Two-Dimensional Self-attention Mechanism

Distillation Enhanced Time Series Forecasting Network with Momentum Contrastive Learning

TS-MVP: Time-Series Representation Learning by Multi-view Prototypical Contrastive Learning.

TS-TWC: A time series representation learning framework based on Time-Wavelet contrasting

Ranking Neighborhood and Class Prototype Contrastive Learning for Time Series

Masking Augmentation for Supervised Learning