Abstract:This paper addresses the task of unsupervised multiobject video segmentation. Most current approaches cast the task as a re-identification solution, which associates objects across frames by generic feature matching. However, the generic features are not reliable for characterizing unseen objects, leading to poor generalization. To address this, we complement current video object segmentation architectures with a discriminative appearance model, capable of capturing more fine-grained target-specific information. Given object proposals from off-the-shelf detectors, three essential strategies are adopted to achieve accurate segmentation: 1) Target-specific tracking. Each determined target is sequentially tracked using a memory-augmented appearance model, wherein the memory stores historical information for re-training the appearance model online; 2) Target-agnostic verification. The tracked segments and object proposals are backward re-identified to trace possible tracklets. Departing from the tradition of only matching proposals between adjacency frames, we conduct long-term semantic matching among distant proposals. This helps to correct the inaccurate tracked segments or drifted results; 3) Adaptive memory updating. Memories are adaptively updated using the verified segments, instead of using tracked results all the time. This favors storing high-quality target information in the memory, reducing the risk for model drifting. By these carefully designs, our approach obtains state-of-the-art performance on DAVIS20 test-dev set (J&F: 59.8%) with a fast speed (15 FPS). It finally ranked 2 place in the DAVIS20 Unsupervised Segmentation Challenge (test-challenge set).

Discovery of Topical Objects from Video: A Structured Dictionary Learning Approach

Discovery of Topical Object in Image Collections

Discovery of the Topical Object in Commercial Video: A Sparse Coding Method

Multilevel Spatial-Temporal Feature Aggregation for Video Object Detection

Discover and Learn New Objects from Documentaries

Non-rigid Video Object Segmentation Based on Semantic Multi-level Framework.

Learning Universal Shape Dictionary for Realtime Instance Segmentation

Target-Aware Object Discovery and Association for Unsupervised Video Multi-Object Segmentation

DEBUG: A Dense Bottom-Up Grounding Approach for Natural Language Video Localization.

Exploiting Spatial-Temporal Locality of Tracking Via Structured Dictionary Learning

Topology dictionary for 3D video understanding

A Combing Top-Down And Bottom-Up Discriminative Dictionaries Learning For Non-Specific Object Detection

Target Aware Adaptive Tracking for Unsupervised Video Object Segmentation

Online Data Organizer: Micro-Video Categorization by Structure-Guided Multimodal Dictionary Learning.

Unsupervised Object Discovery Via Object-Centric Representation

Separable Structure Modeling for Semi-Supervised Video Object Segmentation

Look Further to Recognize Better: Learning Shared Topics and Category-Specific Dictionaries for Open-Ended 3D Object Recognition

Adaptive Selection of Reference Frames for Video Object Segmentation.

Motion-inductive Self-supervised Object Discovery in Videos

Described Spatial-Temporal Video Detection

Reasoning-Enhanced Object-Centric Learning for Videos