Abstract:Time series are usually rarely or sparsely labeled, which limits the performance of deep learning models. Self-supervised representation learning can reduce the reliance of deep learning models on labeled data by extracting structure and feature information from unlabeled data and improve model performance when labeled data is insufficient. Although SimCLR has achieved impressive success in the computer vision field, direct applying SimCLR to time series field usually performs poorly due to the part of data augmentation and the part of feature extractor not being adapted to the temporal dependencies within the time series data. In order to obtain high-quality time series representations, we propose TimeCLR, a framework which is suitable for univariate time series representation, by combining the advantages of DTW and InceptionTime. Inspired by the DTW-based k-nearest neighbor classifier, we first propose the DTW data augmentation that can generate DTW-targeted phase shift and amplitude change phenomena and retain time series structure and feature information. Inspired by the current state-of-the-art deep learning-based time series classification method, InceptionTime, which has good feature extraction capabilities, we designed a feature extractor capable of generating representations in an end-to-end manner. Finally, combining the advantages of DTW data augmentation and InceptionTime, our proposed TimeCLR method successfully extends SimCLR and applies it to the time series field. We designed a variety of experiments and performed careful ablation studies. Experimental results show that our proposed TimeCLR method can not only achieve comparable performance to supervised InceptionTime on multiple tasks, but also produce better performance than supervised learning models in the case of insufficient labeled data, and can be flexibly applied to univariate time series data from different domains.

Self-Supervised Pre-training for Time Series Classification

Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series

Time Series Feature Learning with Labeled and Unlabeled Data

DCS: Debiased Contrastive Learning with Weak Supervision for Time Series Classification

NuTime: Numerically Multi-Scaled Embedding for Large-Scale Time-Series Pretraining

A Deep Multi-Task Representation Learning Method for Time Series Classification and Retrieval.

Rethinking self-supervised learning for time series forecasting: A temporal perspective

Contrastive learning based self-supervised time-series analysis

TimeCLR: A self-supervised contrastive learning framework for univariate time series representation

Toward a Foundation Model for Time Series Data

A Systematic Evaluation of Generated Time Series and Their Effects in Self-Supervised Pretraining

Self-Distilled Representation Learning for Time Series

Learning Transferable Time Series Classifier with Cross-Domain Pre-training from Language Model.

A Survey on Time-Series Pre-Trained Models

Self-Attention Causal Dilated Convolutional Neural Network for Multivariate Time Series Classification and Its Application

Self-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification

Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline

Self-labeling techniques for semi-supervised time series classification: an empirical study

TimeNet: Pre-trained deep recurrent neural network for time series classification

Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects