Abstract:Deep visual graph matching (GM) is a challenging combinatorial task that involves finding a permutation matrix that indicates the correspondence between keypoints from a pair of images. Like many learning systems, empirical studies have shown that visual GM is susceptible to adversarial attacks, with reliability issues in downstream applications. To the best of our knowledge, certifying robustness for deep visual GM remains an open challenge with two main difficulties: how to handle the paired inputs together with the heavily non-linear permutation output space (especially at large scale), and how to balance the trade-off between certified robustness and matching performance. Inspired by the randomized smoothing (RS) technique, we propose the Certified Robustness based on the Optimal Smoothing Range Search (CR-OSRS) technique to fulfill the robustness guarantee for deep visual GM. First, unlike conventional RS methods that use isotropic Gaussian distributions for smoothing, we build the smoothed model with paired joint Gaussian distributions, which capture the structural information among keypoints, and mitigate the performance degradation caused by smoothing. For the vast space of the permutation output, we devise a similarity-based partitioning method that can lower the computational complexity and certification difficulty. We then derive a stringent robustness guarantee that links the certified space of inputs to their corresponding fixed outputs. Second, we design a global optimization method to search for optimal joint Gaussian distributions and facilitate a larger certified space and better performance. Third, we apply data augmentation and a similarity-based regularizer in training to enhance smoothed model performance. Lastly, for the high-dimensional and multivariable nature of the certified space, we propose two methods (sampling and marginal radii) to evaluate it. Experimental results on public benchmarks show that our method achieves state-of-the-art certified robustness.

Certified Robustness on Visual Graph Matching Via Searching Optimal Smoothing Range

Appearance and Structure Aware Robust Deep Visual Graph Matching: Attack, Defense and Beyond

Leveraging Local Planar Motion Property for Robust Visual Matching and Localization.

RGM: A Robust Generalizable Matching Model

GSmooth: Certified Robustness against Semantic Transformations via Generalized Randomized Smoothing

RGM: A Robust Generalist Matching Model.

Robustness Certification of Visual Perception Models via Camera Motion Smoothing

Pixel-wise Smoothing for Certified Robustness against Camera Motion Perturbations

Robust Image Matching Via Local Graph Structure Consensus

Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness

Robust Outlier Removal Using Penalized Linear Regression in Multiview Geometry

Progressive Probabilistic Graph Matching With Local Consistency Regularization

Center Smoothing: Certified Robustness for Networks with Structured Outputs

Joint Transformation Learning Via the L2,1-Norm Metric for Robust Graph Matching.

Spatially Coherent Matching for Robust Registration

Robust Feature Matching Via Graph Neighborhood Motion Consensus

General Lipschitz: Certified Robustness Against Resolvable Semantic Transformations via Transformation-Dependent Randomized Smoothing

Joint Transformation Learning via the L 2,1 -Norm Metric for Robust Graph Matching

Smoothness-Driven Consensus Based on Compact Representation for Robust Feature Matching

Certified Adversarial Robustness via Partition-based Randomized Smoothing

Generalized Correspondence Matching via Flexible Hierarchical Refinement and Patch Descriptor Distillation