Target Aware Adaptive Tracking for Unsupervised Video Object Segmentation

Tianfei Zhou,Wenguan Wang,Yazhou Yao,Jianbing Shen

2020-01-01

Abstract:This paper addresses the task of unsupervised multiobject video segmentation. Most current approaches cast the task as a re-identification solution, which associates objects across frames by generic feature matching. However, the generic features are not reliable for characterizing unseen objects, leading to poor generalization. To address this, we complement current video object segmentation architectures with a discriminative appearance model, capable of capturing more fine-grained target-specific information. Given object proposals from off-the-shelf detectors, three essential strategies are adopted to achieve accurate segmentation: 1) Target-specific tracking. Each determined target is sequentially tracked using a memory-augmented appearance model, wherein the memory stores historical information for re-training the appearance model online; 2) Target-agnostic verification. The tracked segments and object proposals are backward re-identified to trace possible tracklets. Departing from the tradition of only matching proposals between adjacency frames, we conduct long-term semantic matching among distant proposals. This helps to correct the inaccurate tracked segments or drifted results; 3) Adaptive memory updating. Memories are adaptively updated using the verified segments, instead of using tracked results all the time. This favors storing high-quality target information in the memory, reducing the risk for model drifting. By these carefully designs, our approach obtains state-of-the-art performance on DAVIS20 test-dev set (J&F: 59.8%) with a fast speed (15 FPS). It finally ranked 2 place in the DAVIS20 Unsupervised Segmentation Challenge (test-challenge set).