Abstract:While supervised learning approaches show great vitality and effectiveness in video object segmentation, most of them require large amounts of annotations which are expensive and time-consuming. Recently, self-supervised learning has attracted great attention by benefiting from unlabeled video sequences. However, current patch-based self-supervised video object segmentation methods only discriminate the patch from the entire image without distinguishing the object of interest from meaningless backgrounds or even occlusion. These disturbances deteriorate the extracted features and hinder the robustness of tracking when applied to real-world video sequences. In this paper, we propose a novel model named Tracker With Integration-Augmented Attention (TWIAA) to achieve both label-free and prominent performance. Specifically, we integrate both spatial and channel dimensions by introducing a feature spatial enhancement module and a two-stream channel module. With the combination of the two modules, the network can focus on exploring the discriminative object and suppressing the irrelevant part to improve the tracking robustness. Moreover, unlike other methods that calculate features separately on the search branch and template branch, the two designed modules coupled with the Siamese network compute the respective features of the search branch and the template branch jointly to augment the interdependence of the two branches. Such interdependence is injected into both spatial and channel dimensions. So that our approach establishes richer and more discriminative associations to identify the object more accurately. In addition, our method takes full advantage of cycle-consistency information in consecutive frames, which uses coherence as the learning signal to acquire object-oriented relationships. Extensive experiments and ablation studies are conducted on large VOS benchmarks, including DAVIS-2017, YouTube-VOS-2018, and YouTube-VOS-2019. The results verify that our proposed framework has both strong feature representation and competitive performance compared with supervised and self-supervised models.

Random-sampling-based spatial-temporal feature for consumer video concept classification

Self-supervised Video Object Segmentation Using Integration-Augmented Attention

Video Concept Detection Using Support Vector Machines - TRECVID 2007 Evaluations

Learning Concept Bundles for Video Search with Complex Queries

Semantic Video Classification And Feature Subset Selection Under Context And Concept Uncertainty

Automatic And Robust Classification Of Independent Motions In Video Sequences

Consumer video retargeting: context assisted spatial-temporal grid optimization.

Robust Semantic Concept Detection in Large Video Collections

Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification

Spatial-Temporal Multi-level Association for Video Object Segmentation

TV Ad Video Categorization with Probabilistic Latent Concept Learning

Utilizing Related Samples to Learn Complex Queries in Interactive Concept-Based Video Search

Concepts Not Alone: Exploring Pairwise Relationships for Zero-Shot Video Activity Recognition

Incremetal Spatio-Temporal Feature Extraction and Retrieval for Large Video Database

Zero-Shot Video Event Detection With High-Order Semantic Concept Discovery and Matching

Utilizing Related Samples to Enhance Interactive Concept-Based Video Search

Web Video Categorization Using Category-Predictive Classifiers and Category-Specific Concept Classifiers

Building A Large Concept Bank for Representing Events in Video

Video search in concept subspace: a text-like paradigm

Modeling Temporal Concept Receptive Field Dynamically for Untrimmed Video Analysis

Video diver: generic video indexing with diverse features.