Abstract:Graph Neural Networks (GNNs) have gained popularity in image matching methods, proving useful for various computer vision tasks like Structure from Motion (SfM) and 3D reconstruction. A well-known example is SuperGlue. Lightweight variants, such as LightGlue, have been developed with a focus on stacking fewer GNN layers compared to SuperGlue. This paper proposes the h-GNN, a lightweight image matching model, with improvements in the two processing modules, the GNN and matching modules. After image features are detected and described as keypoint nodes of a base graph, the GNN module, which primarily aims at increasing the h-GNN’s depth, creates successive hierarchies of compressed-size graphs from the base graph through a clustering technique termed SC+PCA. SC+PCA combines Principal Component Analysis (PCA) with Spectral Clustering (SC) to enrich nodes with local and global information during graph clustering. A dual non-contrastive clustering loss is used to optimize graph clustering. Additionally, four message-passing mechanisms have been proposed to only update node representations within a graph cluster at the same hierarchical level or to update node representations across graph clusters at different hierarchical levels. The matching module performs iterative pairwise matching on the enriched node representations to obtain a scoring matrix. This matrix comprises scores indicating potential correct matches between the image keypoint nodes. The score matrix is refined with a ‘dustbin’ to further suppress unmatched features. There is a reprojection loss used to optimize keypoint match positions. The Sinkhorn algorithm generates a final partial assignment from the refined score matrix. Experimental results demonstrate the performance of the proposed h-GNN against competing state-of-the-art (SOTA) GNN-based methods on several image matching tasks under homography, estimation, indoor and outdoor camera pose estimation, and 3D reconstruction on multiple datasets. Experiments also demonstrate improved computational memory and runtime, approximately 38.1% and 26.14% lower than SuperGlue, and an average of about 6.8% and 7.1% lower than LightGlue. Future research will explore the effects of integrating more recent simplicial message-passing mechanisms, which concurrently update both node and edge representations, into our proposed model.

Hierarchical Graph Matching Network for Graph Similarity Computation

H2MN: Graph Similarity Learning with Hierarchical Hypergraph Matching Networks.

Graph Partitioning and Graph Neural Network Based Hierarchical Graph Matching for Graph Similarity Computation

Multilevel Graph Matching Networks for Deep Graph Similarity Learning

Hierarchical Large-scale Graph Similarity Computation via Graph Coarsening and Matching

Multi-level Graph Matching Networks for Deep and Robust Graph Similarity Learning

Hierarchical and Fast Graph Similarity Computation Via Graph Coarsening and Deep Graph Learning.

Graph Edit Distance Computation Via Graph Neural Networks.

Learning-based Efficient Graph Similarity Computation via Multi-Scale Convolutional Set Matching

CGMN: A Contrastive Graph Matching Network for Self-Supervised Graph Similarity Learning

SimGNN: A Neural Network Approach to Fast Graph Similarity Computation

Computing Graph Edit Distance via Neural Graph Matching

SEGMN: A Structure-Enhanced Graph Matching Network for Graph Similarity Learning

Graph Similarity Computation via Interpretable Neural Node Alignment

Hierarchical Graph Neural Network: A Lightweight Image Matching Model with Enhanced Message Passing of Local and Global Information in Hierarchical Graph Neural Networks

Convolutional Set Matching for Graph Similarity

Sub-GMN: The Neural Subgraph Matching Network Model

End-to-End Learning of Graph Similarity

Graph Convolutional Network Based on Manifold Similarity Learning