Abstract:Time series are usually rarely or sparsely labeled, which limits the performance of deep learning models. Self-supervised representation learning can reduce the reliance of deep learning models on labeled data by extracting structure and feature information from unlabeled data and improve model performance when labeled data is insufficient. Although SimCLR has achieved impressive success in the computer vision field, direct applying SimCLR to time series field usually performs poorly due to the part of data augmentation and the part of feature extractor not being adapted to the temporal dependencies within the time series data. In order to obtain high-quality time series representations, we propose TimeCLR, a framework which is suitable for univariate time series representation, by combining the advantages of DTW and InceptionTime. Inspired by the DTW-based k-nearest neighbor classifier, we first propose the DTW data augmentation that can generate DTW-targeted phase shift and amplitude change phenomena and retain time series structure and feature information. Inspired by the current state-of-the-art deep learning-based time series classification method, InceptionTime, which has good feature extraction capabilities, we designed a feature extractor capable of generating representations in an end-to-end manner. Finally, combining the advantages of DTW data augmentation and InceptionTime, our proposed TimeCLR method successfully extends SimCLR and applies it to the time series field. We designed a variety of experiments and performed careful ablation studies. Experimental results show that our proposed TimeCLR method can not only achieve comparable performance to supervised InceptionTime on multiple tasks, but also produce better performance than supervised learning models in the case of insufficient labeled data, and can be flexibly applied to univariate time series data from different domains.

Ranking Neighborhood and Class Prototype Contrastive Learning for Time Series

Towards Unsupervised Time Series Representation Learning: A Decomposition Perspective

Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series

DCS: Debiased Contrastive Learning with Weak Supervision for Time Series Classification

Rank Supervised Contrastive Learning for Time Series Classification

Contrastive Representation Based Active Learning for Time Series

Self-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification

Time-Series Contrastive Learning against False Negatives and Class Imbalance

REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning

Semi-Supervised End-To-End Contrastive Learning For Time Series Classification

Time-Series Representation Learning via Temporal and Contextual Contrasting

Multi-Task Self-Supervised Time-Series Representation Learning

What Constitutes Good Contrastive Learning in Time-Series Forecasting?

TimeCLR: A self-supervised contrastive learning framework for univariate time series representation

Supervised Contrastive Few-Shot Learning for High-Frequency Time Series

RankCSE: Unsupervised Sentence Representations Learning Via Learning to Rank

LTCR: Long Temporal Characteristic Reconstruction for Segmentation in Contrastive Learning

Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Series