Self-supervised Learning of Visual Graph Matching

Chang Liu,Shaofeng Zhang,Xiaokang Yang,Junchi Yan
DOI: https://doi.org/10.1007/978-3-031-20050-2_22
2022-01-01
Abstract:Despite the rapid progress made by existing graph matching methods, expensive or even unrealistic node-level correspondence labels are often required. Inspired by recent progress in self-supervised contrastive learning, we propose an end-to-end label-free self-supervised contrastive graph matching framework (SCGM). Unlike in vision tasks like classification and segmentation, where the backbone is often forced to extract object instance-level or pixel-level information, we design an extra objective function at node-level on graph data which also considers both the visual appearance and graph structure by node embedding. Further, we propose two-stage augmentation functions on both raw images and extracted graphs to increase the variance, which has been shown effective in self-supervised learning. We conduct experiments on standard graph matching benchmarks, where our method boosts previous stateof-the-arts under both label-free self-supervised and fine-tune settings. Without the ground truth labels for node matching nor the graph/imagelevel category information, our proposed framework SCGM outperforms several deep graph matching methods. By proper fine-tuning, SCGM can surpass the state-of-the-art supervised deep graph matching methods. Code is available at https://github.com/Thinklab-SJTU/ThinkMatchSCGM.
What problem does this paper attempt to address?