Abstract:While supervised learning approaches show great vitality and effectiveness in video object segmentation, most of them require large amounts of annotations which are expensive and time-consuming. Recently, self-supervised learning has attracted great attention by benefiting from unlabeled video sequences. However, current patch-based self-supervised video object segmentation methods only discriminate the patch from the entire image without distinguishing the object of interest from meaningless backgrounds or even occlusion. These disturbances deteriorate the extracted features and hinder the robustness of tracking when applied to real-world video sequences. In this paper, we propose a novel model named Tracker With Integration-Augmented Attention (TWIAA) to achieve both label-free and prominent performance. Specifically, we integrate both spatial and channel dimensions by introducing a feature spatial enhancement module and a two-stream channel module. With the combination of the two modules, the network can focus on exploring the discriminative object and suppressing the irrelevant part to improve the tracking robustness. Moreover, unlike other methods that calculate features separately on the search branch and template branch, the two designed modules coupled with the Siamese network compute the respective features of the search branch and the template branch jointly to augment the interdependence of the two branches. Such interdependence is injected into both spatial and channel dimensions. So that our approach establishes richer and more discriminative associations to identify the object more accurately. In addition, our method takes full advantage of cycle-consistency information in consecutive frames, which uses coherence as the learning signal to acquire object-oriented relationships. Extensive experiments and ablation studies are conducted on large VOS benchmarks, including DAVIS-2017, YouTube-VOS-2018, and YouTube-VOS-2019. The results verify that our proposed framework has both strong feature representation and competitive performance compared with supervised and self-supervised models.

Joint Multisource Saliency and Exemplar Mechanism for Weakly Supervised Video Object Segmentation.

Learning Spatiotemporal Relationships with a Unified Framework for Video Object Segmentation

Weakly Supervised Video Object Segmentation via Dual-attention Cross-branch Fusion

Self-supervised Video Object Segmentation Using Integration-Augmented Attention

Segmentation in Weakly Labeled Videos via a Semantic Ranking and Optical Warping Network

Exploring Weakly Labeled Images for Video Object Segmentation with Submodular Proposal Selection

Multidimensional Exploration of Segment Anything Model for Weakly Supervised Video Salient Object Detection

Learning Spatial-Semantic Features for Robust Video Object Segmentation

Joint Attention Mechanism for Unsupervised Video Object Segmentation

Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping

Object-based spatial similarity for semi-supervised video object segmentation

Video Object Segmentation by Multi-Scale Attention Using Bidirectional Strategy

OWS-Seg: Online Weakly Supervised Video Instance Segmentation Via Contrastive Learning

Saliency-Aware Video Object Segmentation.

Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation.

Learning Unsupervised Video Object Segmentation Through Visual Attention

Superpixel Consistency Saliency Map Generation for Weakly Supervised Semantic Segmentation of Remote Sensing Images

Joint Modeling of Feature, Correspondence, and a Compressed Memory for Video Object Segmentation

Spatial-Temporal Multi-level Association for Video Object Segmentation

Tracking-assisted Weakly Supervised Online Visual Object Segmentation in Unconstrained Videos

Unified Multi-Modality Video Object Segmentation Using Reinforcement Learning