Abstract:Video self-supervised learning is a challenging task, which requires significant expressive power from the model to leverage rich spatial-temporal knowledge and generate effective supervisory signals from large amounts of unlabeled videos. However, existing methods fail to increase the temporal diversity of unlabeled videos and ignore elaborately modeling multi-scale temporal dependencies in an explicit way. To overcome these limitations, we take advantage of the multi-scale temporal dependencies within videos and proposes a novel video self-supervised learning framework named Temporal Contrastive Graph Learning (TCGL), which jointly models the inter-snippet and intra-snippet temporal dependencies for temporal representation learning with a hybrid graph contrastive learning strategy. Specifically, a Spatial-Temporal Knowledge Discovering (STKD) module is first introduced to extract motion-enhanced spatial-temporal representations from videos based on the frequency domain analysis of discrete cosine transform. To explicitly model multi-scale temporal dependencies of unlabeled videos, our TCGL integrates the prior knowledge about the frame and snippet orders into graph structures, i.e., the intra-/inter- snippet Temporal Contrastive Graphs (TCG). Then, specific contrastive learning modules are designed to maximize the agreement between nodes in different graph views. To generate supervisory signals for unlabeled videos, we introduce an Adaptive Snippet Order Prediction (ASOP) module which leverages the relational knowledge among video snippets to learn the global context representation and recalibrate the channel-wise features adaptively. Experimental results demonstrate the superiority of our TCGL over the state-of-the-art methods on large-scale action recognition and video retrieval <a class="link-external link-http" href="http://benchmarks.The" rel="external noopener nofollow">this http URL</a> code is publicly available at <a class="link-external link-https" href="https://github.com/YangLiu9208/TCGL" rel="external noopener nofollow">this https URL</a>.

Multi-view Self-Supervised Contrastive Learning for Multivariate Time Series

Self-supervised Contrastive Representation Learning for Semi-supervised Time-Series Classification

Towards Unsupervised Time Series Representation Learning: A Decomposition Perspective

DCS: Debiased Contrastive Learning with Weak Supervision for Time Series Classification

Time-Series Representation Learning via Temporal and Contextual Contrasting

Time-Series Forecasting Through Contrastive Learning with a Two-Dimensional Self-attention Mechanism

Multi-Task Self-Supervised Time-Series Representation Learning

What Constitutes Good Contrastive Learning in Time-Series Forecasting?

Multi-view self-supervised learning for multivariate variable-channel time series

Supervised Contrastive Few-Shot Learning for High-Frequency Time Series

TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning

Time-series representation learning via Time-Frequency Fusion Contrasting

Self-supervised Representation Learning for Time Series Via Temporal Contrasting and Transformation

FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space

Can Temporal Information Help with Contrastive Self-Supervised Learning?

Time Series Representation Learning with Supervised Contrastive Temporal Transformer

TCHA: Contrastive Learning Via Hybrid Granularities and Adaptive Sampling for Multivariate Time Series

TimeCLR: A self-supervised contrastive learning framework for univariate time series representation