Graph Matching with Bi-level Noisy Correspondence

Yijie Lin,Mouxing Yang,Jun Yu,Peng Hu,Changqing Zhang,Xi Peng

2023-08-05

Abstract:In this paper, we study a novel and widely existing problem in graph matching (GM), namely, Bi-level Noisy Correspondence (BNC), which refers to node-level noisy correspondence (NNC) and edge-level noisy correspondence (ENC). In brief, on the one hand, due to the poor recognizability and viewpoint differences between images, it is inevitable to inaccurately annotate some keypoints with offset and confusion, leading to the mismatch between two associated nodes, i.e., NNC. On the other hand, the noisy node-to-node correspondence will further contaminate the edge-to-edge correspondence, thus leading to ENC. For the BNC challenge, we propose a novel method termed Contrastive Matching with Momentum Distillation. Specifically, the proposed method is with a robust quadratic contrastive loss which enjoys the following merits: i) better exploring the node-to-node and edge-to-edge correlations through a GM customized quadratic contrastive learning paradigm; ii) adaptively penalizing the noisy assignments based on the confidence estimated by the momentum teacher. Extensive experiments on three real-world datasets show the robustness of our model compared with 12 competitive baselines. The code is available at <a class="link-external link-https" href="https://github.com/XLearning-SCU/2023-ICCV-COMMON" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper aims to address a new challenge in Graph Matching (GM) — Bi-level Noisy Correspondence (BNC). Specifically:

1. **Problem Background**: In practical applications, due to insufficient image recognition capabilities and perspective differences, manually annotated key points often exhibit deviations or errors, leading to inaccurate matches between nodes (i.e., Node-level Noisy Correspondence, NNC), which in turn affects the matching between edges (i.e., Edge-level Noisy Correspondence, ENC).

2. **Research Objective**: Propose a new method to tackle this challenge, namely **COntrastive Matching with MOmentum distillatioN (COMMON)**, to improve the robustness of graph matching under noisy conditions.

3. **Solution**:
- **Contrastive Loss**: Designed a contrastive learning loss function that includes linear and quadratic geometric information, exploring the correlation between nodes and edges through two simple graph consistency regularization terms.
- **Momentum Distillation**: Utilized a momentum encoder to generate high-quality pseudo-labels, dynamically adjusting noisy correspondences to enhance the model's robustness to noise.

4. **Experimental Validation**: Extensive experiments on three real-world datasets validated the effectiveness of the proposed method, demonstrating outstanding performance in multiple benchmarks.

In summary, the main contribution of this paper lies in revealing a critical issue in graph matching and proposing an effective solution, thereby significantly improving the performance of graph matching algorithms in the presence of noise.