Self Supervised Progressive Network for High Performance Video Object Segmentation
Guorong Li,Dexiang Hong,Kai Xu,Bineng Zhong,Li Su,Zhenjun Han,Qingming Huang
DOI: https://doi.org/10.1109/tnnls.2022.3219936
IF: 14.255
2022-01-01
IEEE Transactions on Neural Networks and Learning Systems
Abstract:Recently, self-supervised video object segmentation (VOS) has attracted much interest. However, most proxy tasks are proposed to train only a single backbone, which relies on a point-to-point correspondence strategy to propagate masks through a video sequence. Due to its simple pipeline, the performance of the single backbone paradigm is still unsatisfactory. Instead of following the previous literature, we propose our self-supervised progressive network (SSPNet) which consists of a memory retrieval module (MRM) and collaborative refinement module (CRM). The MRM can perform point-to-point correspondence and produce a propagated coarse mask for a query frame through self-supervised pixel-level and frame-level similarity learning. The CRM, which is trained via cycle consistency region tracking, aggregates the reference & query information and learns the collaborative relationship among them implicitly to refine the coarse mask. Furthermore, to learn semantic knowledge from unlabeled data, we also design two novel mask-generation strategies to provide the training data with meaningful semantic information for the CRM. Extensive experiments conducted on DAVIS-17, YouTube-VOS and SegTrack v2 demonstrate that our method surpasses the state-of-the-art self-supervised methods and narrows the gap with the fully supervised methods.
computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture