Abstract:Self-supervised contrastive learning has become a key technique in deep learning, particularly in time series analysis, due to its ability to learn meaningful representations without explicit supervision. Augmentation is a critical component in contrastive learning, where different augmentations can dramatically impact performance, sometimes influencing accuracy by over 30%. However, the selection of augmentations is predominantly empirical which can be suboptimal, or grid searching that is time-consuming. In this paper, we establish a principled framework for selecting augmentations based on dataset characteristics such as trend and seasonality. Specifically, we construct 12 synthetic datasets incorporating trend, seasonality, and integration weights. We then evaluate the effectiveness of 8 different augmentations across these synthetic datasets, thereby inducing generalizable associations between time series characteristics and augmentation efficiency. Additionally, we evaluated the induced associations across 6 real-world datasets encompassing domains such as activity recognition, disease diagnosis, traffic monitoring, electricity usage, mechanical fault prognosis, and finance. These real-world datasets are diverse, covering a range from 1 to 12 channels, 2 to 10 classes, sequence lengths of 14 to 1280, and data frequencies from 250 Hz to daily intervals. The experimental results show that our proposed trend-seasonality-based augmentation recommendation algorithm can accurately identify the effective augmentations for a given time series dataset, achieving an average Recall@3 of 0.667, outperforming baselines. Our work provides guidance for studies employing contrastive learning in time series analysis, with wide-ranging applications. All the code, datasets, and analysis results will be released at <a class="link-external link-https" href="https://github.com/DL4mHealth/TS-Contrastive-Augmentation-Recommendation" rel="external noopener nofollow">this https URL</a>.

Interactive Augmentations, Features, and Parameters for Contrastive Learning

Boosting Unsupervised Contrastive Learning Using Diffusion-Based Data Augmentation from Scratch

MetAug: Contrastive Learning via Meta Feature Augmentation

SimGRACE: A Simple Framework for Graph Contrastive Learning Without Data Augmentation

Time Series Contrastive Learning with Information-Aware Augmentations

Feature Augmentation for Self-supervised Contrastive Learning: A Closer Look

Contrastive Learning With Stronger Augmentations

Adaptive Data Augmentation for Contrastive Learning

Graph Contrastive Learning with Adaptive Augmentation

Full-Attention Driven Graph Contrastive Learning: with Effective Mutual Information Insight

Parametric Augmentation for Time Series Contrastive Learning

CLAP: Isolating Content from Style through Contrastive Learning with Augmented Prompts

Rethinking the Augmentation Module in Contrastive Learning: Learning Hierarchical Augmentation Invariance with Expanded Views

Contrastive learning with text augmentation for text classification

Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning

Guidelines for Augmentation Selection in Contrastive Learning for Time Series Classification

Differentiable Data Augmentation for Contrastive Sentence Representation Learning

Multi-Scale Subgraph Contrastive Learning

ScoreCL: Augmentation-Adaptive Contrastive Learning via Score-Matching Function

Adaptive Multi-head Contrastive Learning

Unsupervised Document Embedding via Contrastive Augmentation