Abstract:In recent years, the introduction of self-supervised contrastive learning (SSCL) has demonstrated remarkable improvements in representation learning across various domains, including natural language processing and computer vision. By leveraging the inherent benefits of self-supervision, SSCL enables the pre-training of representation models using vast amounts of unlabeled data. Despite these advances, there remains a significant gap in understanding the impact of different SSCL strategies on time series forecasting performance, as well as the specific benefits that SSCL can bring. This paper aims to address these gaps by conducting a comprehensive analysis of the effectiveness of various training variables, including different SSCL algorithms, learning strategies, model architectures, and their interplay. Additionally, to gain deeper insights into the improvements brought about by SSCL in the context of time-series forecasting, a qualitative analysis of the empirical receptive field is performed. Through our experiments, we demonstrate that the end-to-end training of a Transformer model using the Mean Squared Error (MSE) loss and SSCL emerges as the most effective approach in time series forecasting. Notably, the incorporation of the contrastive objective enables the model to prioritize more pertinent information for forecasting, such as scale and periodic relationships. These findings contribute to a better understanding of the benefits of SSCL in time series forecasting and provide valuable insights for future research in this area. Our codes are available at <a class="link-external link-https" href="https://github.com/chiyuzhang94/contrastive_learning_time-series_e2e" rel="external noopener nofollow">this https URL</a>.

Identifying Erroneous Software Changes through Self-Supervised Contrastive Learning on Time Series Data

Hierarchical Semi-supervised Contrastive Learning for Contamination-Resistant Anomaly Detection

Skip-Step Contrastive Predictive Coding for Time Series Anomaly Detection.

DCS: Debiased Contrastive Learning with Weak Supervision for Time Series Classification

Learning Discrimination from Contaminated Data: Multi-Instance Learning for Unsupervised Anomaly Detection

Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series

USD: Unsupervised Soft Contrastive Learning for Fault Detection in Multivariate Time Series

Identifying Bad Software Changes Via Multimodal Anomaly Detection for Online Service Systems

Time-Series Contrastive Learning against False Negatives and Class Imbalance

Unsupervised CD in Satellite Image Time Series by Contrastive Learning and Feature Tracking

Contrastive learning based self-supervised time-series analysis

Contrastive-based Bi-update Class Incremental Model and Its Application in Fault Diagnosis

A self-supervised contrastive change point detection method for industrial time series

Time-series Anomaly Detection via Contextual Discriminative Contrastive Learning

HCL-MTSAD: Hierarchical Contrastive Consistency Learning for Accurate Detection of Industrial Multivariate Time Series Anomalies

Self-Supervised Time-Series Anomaly Detection Using Learnable Data Augmentation

Time Series Contrastive Learning with Information-Aware Augmentations

Debiased Contrastive Learning for Time-Series Representation Learning and Fault Detection

What Constitutes Good Contrastive Learning in Time-Series Forecasting?

CL-MMAD: A Contrastive Learning Based Multimodal Software Runtime Anomaly Detection Method