Abstract:In recent years, the introduction of self-supervised contrastive learning (SSCL) has demonstrated remarkable improvements in representation learning across various domains, including natural language processing and computer vision. By leveraging the inherent benefits of self-supervision, SSCL enables the pre-training of representation models using vast amounts of unlabeled data. Despite these advances, there remains a significant gap in understanding the impact of different SSCL strategies on time series forecasting performance, as well as the specific benefits that SSCL can bring. This paper aims to address these gaps by conducting a comprehensive analysis of the effectiveness of various training variables, including different SSCL algorithms, learning strategies, model architectures, and their interplay. Additionally, to gain deeper insights into the improvements brought about by SSCL in the context of time-series forecasting, a qualitative analysis of the empirical receptive field is performed. Through our experiments, we demonstrate that the end-to-end training of a Transformer model using the Mean Squared Error (MSE) loss and SSCL emerges as the most effective approach in time series forecasting. Notably, the incorporation of the contrastive objective enables the model to prioritize more pertinent information for forecasting, such as scale and periodic relationships. These findings contribute to a better understanding of the benefits of SSCL in time series forecasting and provide valuable insights for future research in this area. Our codes are available at <a class="link-external link-https" href="https://github.com/chiyuzhang94/contrastive_learning_time-series_e2e" rel="external noopener nofollow">this https URL</a>.

Self-Supervised Contrastive Learning for Long-term Forecasting

Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series

Towards Unsupervised Time Series Representation Learning: A Decomposition Perspective

What Constitutes Good Contrastive Learning in Time-Series Forecasting?

Distillation Enhanced Time Series Forecasting Network with Momentum Contrastive Learning

When Do Contrastive Learning Signals Help Spatio-Temporal Graph Forecasting?

Contrastive Learning with Hallucinating Data for Long-Tailed Face Recognition.

Self-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification

Contrastive learning based self-supervised time-series analysis

A Contrastive Learning-based Framework for Wind power forecast

CLeaRForecast: Contrastive Learning of High-Purity Representations for Time Series Forecasting

Semi-supervised contrastive learning with decomposition-based data augmentation for time series classification

Contrastive Difference Predictive Coding

Contrastive Neural Processes for Self-Supervised Learning

Simple Contrastive Representation Learning for Time Series Forecasting

LTCR: Long Temporal Characteristic Reconstruction for Segmentation in Contrastive Learning

CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting

Introducing Spectral Attention for Long-Range Dependency in Time Series Forecasting

Multi-Task Self-Supervised Time-Series Representation Learning