Soft Contrastive Learning for Time Series

Seunghan Lee,Taeyoung Park,Kibok Lee

2023-12-27

Abstract:Contrastive learning has shown to be effective to learn representations from time series in a self-supervised way. However, contrasting similar time series instances or values from adjacent timestamps within a time series leads to ignore their inherent correlations, which results in deteriorating the quality of learned representations. To address this issue, we propose SoftCLT, a simple yet effective soft contrastive learning strategy for time series. This is achieved by introducing instance-wise and temporal contrastive loss with soft assignments ranging from zero to one. Specifically, we define soft assignments for 1) instance-wise contrastive loss by the distance between time series on the data space, and 2) temporal contrastive loss by the difference of timestamps. SoftCLT is a plug-and-play method for time series contrastive learning that improves the quality of learned representations without bells and whistles. In experiments, we demonstrate that SoftCLT consistently improves the performance in various downstream tasks including classification, semi-supervised learning, transfer learning, and anomaly detection, showing state-of-the-art performance. Code is available at this repository: https://github.com/seunghan96/softclt.

Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

This paper focuses on self-supervised learning methods for time series, specifically the application of contrastive learning. Traditional contrastive learning methods may overlook the intrinsic correlations between similar instances or neighboring timestamps in time series, thus degrading the quality of the learned representations. To address this issue, the paper proposes Soft Contrastive Learning for Time Series (SoftCLT) by introducing instance-level and time-level soft contrastive losses to consider the similarity between different time series and the similarity between different timestamps within a single time series.

Specifically, SoftCLT defines soft assignments for instance-level contrastive loss using distance measures such as Dynamic Time Warping (DTW), and defines soft assignments for time-level contrastive loss using timestamp differences. This approach can be seen as a generalization of standard contrastive loss, which degenerates to standard contrastive loss when soft assignments are replaced with hard assignments (0 or 1).

The experimental section demonstrates the effectiveness of SoftCLT in tasks such as time series classification, semi-supervised learning, transfer learning, and anomaly detection, showing superior performance compared to existing contrastive learning methods, particularly on the UCR and UEA time series datasets where SoftCLT significantly improves accuracy.

Furthermore, the paper compares different distance metric methods, discusses the computation of soft assignments, and explores the application of SoftCLT to other contrastive learning frameworks. Overall, SoftCLT provides an improved representation learning method for self-supervised learning of time series, enhancing the utilization of the intrinsic structure of time series.