Abstract:Video self-supervised learning is a challenging task, which requires significant expressive power from the model to leverage rich spatial-temporal knowledge and generate effective supervisory signals from large amounts of unlabeled videos. However, existing methods fail to increase the temporal diversity of unlabeled videos and ignore elaborately modeling multi-scale temporal dependencies in an explicit way. To overcome these limitations, we take advantage of the multi-scale temporal dependencies within videos and proposes a novel video self-supervised learning framework named Temporal Contrastive Graph Learning (TCGL), which jointly models the inter-snippet and intra-snippet temporal dependencies for temporal representation learning with a hybrid graph contrastive learning strategy. Specifically, a Spatial-Temporal Knowledge Discovering (STKD) module is first introduced to extract motion-enhanced spatial-temporal representations from videos based on the frequency domain analysis of discrete cosine transform. To explicitly model multi-scale temporal dependencies of unlabeled videos, our TCGL integrates the prior knowledge about the frame and snippet orders into graph structures, i.e., the intra-/inter- snippet Temporal Contrastive Graphs (TCG). Then, specific contrastive learning modules are designed to maximize the agreement between nodes in different graph views. To generate supervisory signals for unlabeled videos, we introduce an Adaptive Snippet Order Prediction (ASOP) module which leverages the relational knowledge among video snippets to learn the global context representation and recalibrate the channel-wise features adaptively. Experimental results demonstrate the superiority of our TCGL over the state-of-the-art methods on large-scale action recognition and video retrieval <a class="link-external link-http" href="http://benchmarks.The" rel="external noopener nofollow">this http URL</a> code is publicly available at <a class="link-external link-https" href="https://github.com/YangLiu9208/TCGL" rel="external noopener nofollow">this https URL</a>.

Dynamic-boosting Attention for Self-Supervised Video Representation Learning

Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos

Temporally-Embedded Self-Supervised Video Representation Learning

Self-Supervised Video Representation Learning in a Heuristic Decoupled Perspective

Self-Supervised Representation Learning for Videos by Segmenting Via Sampling Rate Order Prediction

Self-Supervised Spatiotemporal Learning Via Video Clip Order Prediction.

Explore Video Clip Order With Self-Supervised and Curriculum Learning for Video Applications

Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization

Self-supervised Temporal Discriminative Learning for Video Representation Learning

Self-supervised pretext task collaborative multi-view contrastive learning for video action recognition

Enhancing Motion Visual Cues for Self-Supervised Video Representation Learning

Self-supervised Spatio-temporal Representation Learning for Videos by Predicting Motion and Appearance Statistics

ARVideo: Autoregressive Pretraining for Self-Supervised Video Representation Learning

Self-supervised Spatiotemporal Representation Learning by Exploiting Video Continuity

Self-Supervised Video Representation Learning with Motion-Contrastive Perception

Attentive spatial-temporal contrastive learning for self-supervised video representation

Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion

No More Shortcuts: Realizing the Potential of Temporal Self-Supervision

TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning

Time Is MattEr: Temporal Self-supervision for Video Transformers

Skip-Clip: Self-Supervised Spatiotemporal Representation Learning by Future Clip Order Ranking