Rank Supervised Contrastive Learning for Time Series Classification

Qianying Ren,Dongsheng Luo,Dongjin Song

2024-10-10

Abstract:Recently, various contrastive learning techniques have been developed to categorize time series data and exhibit promising performance. A general paradigm is to utilize appropriate augmentations and construct feasible positive samples such that the encoder can yield robust and discriminative representations by mapping similar data points closer together in the feature space while pushing dissimilar data points farther apart. Despite its efficacy, the fine-grained relative similarity (e.g., rank) information of positive samples is largely ignored, especially when labeled samples are limited. To this end, we present Rank Supervised Contrastive Learning (RankSCL) to perform time series classification. Different from conventional contrastive learning frameworks, RankSCL augments raw data in a targeted way in the embedding space and adopts certain filtering rules to select more informative positive and negative pairs of samples. Moreover, a novel rank loss is developed to assign different weights for different levels of positive samples, enable the encoder to extract the fine-grained information of the same class, and produce a clear boundary among different classes. Thoroughly empirical studies on 128 UCR datasets and 30 UEA datasets demonstrate that the proposed RankSCL can achieve state-of-the-art performance compared to existing baseline methods.

Machine Learning

What problem does this paper attempt to address?

The paper aims to address the problem of time series classification, particularly in situations where labeled samples are limited. It focuses on how to utilize fine-grained relative similarity (e.g., ranking) information among positive samples to improve classification performance. Specifically, the paper proposes a method called Rank Supervised Contrastive Learning (RankSCL), which improves existing techniques through the following points:

1. **Data Augmentation**: Unlike traditional contrastive learning methods that directly augment the original time series data, RankSCL performs targeted data augmentation in the embedding space. This generates more diverse samples, thereby enriching the information of the same category and enhancing the boundaries between different categories.

2. **Ranking Loss Function**: A new ranking loss function is developed, which assigns different weights based on the importance of positive samples. This allows the encoder to extract fine-grained information of similar samples and create clear boundaries between different categories.

3. **Effective Representation Learning**: By leveraging the relative similarity information of positive samples, RankSCL can better understand the underlying landscape of the feature space, resulting in more effective representations that aid in time series classification tasks.

Through experiments on multiple benchmark datasets, RankSCL demonstrates superior performance compared to existing methods, particularly showing significant advantages in handling both multivariate and univariate time series classification.