Abstract:Deep visual graph matching (GM) is a challenging combinatorial task that involves finding a permutation matrix that indicates the correspondence between keypoints from a pair of images. Like many learning systems, empirical studies have shown that visual GM is susceptible to adversarial attacks, with reliability issues in downstream applications. To the best of our knowledge, certifying robustness for deep visual GM remains an open challenge with two main difficulties: how to handle the paired inputs together with the heavily non-linear permutation output space (especially at large scale), and how to balance the trade-off between certified robustness and matching performance. Inspired by the randomized smoothing (RS) technique, we propose the Certified Robustness based on the Optimal Smoothing Range Search (CR-OSRS) technique to fulfill the robustness guarantee for deep visual GM. First, unlike conventional RS methods that use isotropic Gaussian distributions for smoothing, we build the smoothed model with paired joint Gaussian distributions, which capture the structural information among keypoints, and mitigate the performance degradation caused by smoothing. For the vast space of the permutation output, we devise a similarity-based partitioning method that can lower the computational complexity and certification difficulty. We then derive a stringent robustness guarantee that links the certified space of inputs to their corresponding fixed outputs. Second, we design a global optimization method to search for optimal joint Gaussian distributions and facilitate a larger certified space and better performance. Third, we apply data augmentation and a similarity-based regularizer in training to enhance smoothed model performance. Lastly, for the high-dimensional and multivariable nature of the certified space, we propose two methods (sampling and marginal radii) to evaluate it. Experimental results on public benchmarks show that our method achieves state-of-the-art certified robustness.

Joint Transformation Learning via the L 2,1 -Norm Metric for Robust Graph Matching

Joint Transformation Learning Via the L2,1-Norm Metric for Robust Graph Matching.

Dual Calibration Mechanism Based L 2 , p-Norm for Graph Matching

Dual Calibration Mechanism Based <i>L</i><sub>2,</sub> <i><sub>p</sub></i>-Norm for Graph Matching

Partial Gromov-Wasserstein Learning for Partial Graph Matching

Learning Bipartite Graph Matching for Robust Visual Localization.

Joint Optimization for Consistent Multiple Graph Matching

Robust Image Matching Via Local Graph Structure Consensus

Graduated Consistency-Regularized Optimization For Multi-Graph Matching

Progressive Probabilistic Graph Matching With Local Consistency Regularization

Robust Graph Matching Using An Unbalanced Hierarchical Optimal Transport Framework

Revocable Deep Reinforcement Learning with Affinity Regularization for Outlier-Robust Graph Matching

Robust Estimation of Nonrigid Transformation for Point Set Registration

Certified Robustness on Visual Graph Matching Via Searching Optimal Smoothing Range

Learnable Graph Matching: A Practical Paradigm for Data Association

A New Algorithm for Non-Rigid Point Matching Using Geodesic Graph Model

RGM: A Robust Generalizable Matching Model

Certified Robustness on Structural Graph Matching

Appearance and Structure Aware Robust Deep Visual Graph Matching: Attack, Defense and Beyond

Robust Feature Matching Via Graph Neighborhood Motion Consensus

DHOT-GM: Robust Graph Matching Using A Differentiable Hierarchical Optimal Transport Framework