Abstract:While supervised learning approaches show great vitality and effectiveness in video object segmentation, most of them require large amounts of annotations which are expensive and time-consuming. Recently, self-supervised learning has attracted great attention by benefiting from unlabeled video sequences. However, current patch-based self-supervised video object segmentation methods only discriminate the patch from the entire image without distinguishing the object of interest from meaningless backgrounds or even occlusion. These disturbances deteriorate the extracted features and hinder the robustness of tracking when applied to real-world video sequences. In this paper, we propose a novel model named Tracker With Integration-Augmented Attention (TWIAA) to achieve both label-free and prominent performance. Specifically, we integrate both spatial and channel dimensions by introducing a feature spatial enhancement module and a two-stream channel module. With the combination of the two modules, the network can focus on exploring the discriminative object and suppressing the irrelevant part to improve the tracking robustness. Moreover, unlike other methods that calculate features separately on the search branch and template branch, the two designed modules coupled with the Siamese network compute the respective features of the search branch and the template branch jointly to augment the interdependence of the two branches. Such interdependence is injected into both spatial and channel dimensions. So that our approach establishes richer and more discriminative associations to identify the object more accurately. In addition, our method takes full advantage of cycle-consistency information in consecutive frames, which uses coherence as the learning signal to acquire object-oriented relationships. Extensive experiments and ablation studies are conducted on large VOS benchmarks, including DAVIS-2017, YouTube-VOS-2018, and YouTube-VOS-2019. The results verify that our proposed framework has both strong feature representation and competitive performance compared with supervised and self-supervised models.

Unsupervised Teacher-Student Model for Large-Scale Video Retrieval.

Self-supervised Video Retrieval Transformer Network

A Supervised Video Hashing Method Based on a Deep 3D Convolutional Neural Network for Large-Scale Video Retrieval

UATVR: Uncertainty-Adaptive Text-Video Retrieval

Self-supervised Video Object Segmentation Using Integration-Augmented Attention

Self-Supervised Video Hashing Via Bidirectional Transformers.

Unsupervised Variational Video Hashing with 1D-CNN-LSTM Networks.

Learn from Unlabeled Videos for Near-duplicate Video Retrieval

Video Big Data Retrieval Over Media Cloud: A Context-Aware Online Learning Approach

Learning Segment Similarity and Alignment in Large-Scale Content Based Video Retrieval

Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrieval

Learning Fine-grained User Interests for Micro-video Recommendation

Unsupervised Deep Video Hashing via Balanced Code for Large-Scale Video Retrieval

Visual Spatio-temporal Relation-enhanced Network for Cross-modal Text-Video Retrieval

Content-based Video Retrieval Integrating Human Perception.

UVIS: Unsupervised Video Instance Segmentation

Deep Unsupervised Key Frame Extraction for Efficient Video Classification

Unified Embedding Alignment for Open-Vocabulary Video Instance Segmentation

DnS: Distill-and-Select for Efficient and Accurate Video Indexing and Retrieval

Unsupervised Learning From Video With Deep Neural Embeddings

A Generic Framework for Video Annotation Via Semi-Supervised Learning.