Abstract:Local feature matching, which aims to establish the matches between image pairs, is a pivotal component of multiple visual applications. While current transformer-based works exhibit remarkable performance, they mechanically alternate self- and cross-attention in a predetermined order without considering their prioritization, culminating in inadequate enhancement of visual descriptors. Moreover, when calculating attention matrices to integrate global context, current methods only explicitly model the correlation among the feature channels without taking their importance into account, leaving insufficient message propagation. In this work, we develop a dynamic sparse attention perception (DSAP) matcher to tackle the aforementioned issues. To resolve the first issue, DSAP presents a dynamic perception strategy that enables the network to dynamically implement feature enhancement via modifying both forward and backward propagation. During forward propagation, DPS assigns a learnable perception score to each transformer layer and employs an exponential moving average algorithm to calculate the current score. After that, DPS utilizes an indicator function to binarize the score, allowing DSAP to adaptively determine the appropriate utilization of self- or cross-attention at current iteration. During backward propagation, DPS employs a gradient estimator that adjusts the gradient of perception scores, thus rendering them differentiable. To tackle the second issue, DSAP introduces a weighted sparse transformer (WSFormer) that recalibrates attention matrices by concurrently considering both channel importance and channel correlation. WSFormer predicts attention vectors to weight attention matrices while constructing multiple sparse attention matrices to integrate various global messages, thus highlighting informative channels and inhibiting redundant message propagation. Extensive experiments in public datasets and real environment demonstrate that DSAP achieves exceptional performances across various downstream tasks, including relative pose estimation and visual localization. The code is available at https://github.com/mooncake199809/DSAP.

Local Feature Matching with Wavelet Attention Transformer

Dissolution of ionizable drugs in buffered and unbuffered solution.

A Local Feature with Multiple Line Descriptors and Its Speeded-Up Matching Algorithm

LoFLAT: Local Feature Matching using Focused Linear Attention Transformer

Learning Geometric Feature Embedding with Transformers for Image Matching

LGFCTR: Local and Global Feature Convolutional Transformer for Image Matching

Adaptive Spot-Guided Transformer for Consistent Local Feature Matching.

LoFTR: Detector-Free Local Feature Matching with Transformers

TKwinFormer: Top k Window Attention in Vision Transformers for Feature Matching

Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence

Semi-Dense Feature Matching with Transformers and Its Applications in Multiple-View Geometry

ETO:Efficient Transformer-based Local Feature Matching by Organizing Multiple Homography Hypotheses

Multi-scale Matching Networks for Semantic Correspondence

Attention Weighted Local Descriptors

DSAP: Dynamic Sparse Attention Perception Matcher for Accurate Local Feature Matching

MTLDesc: Looking Wider to Describe Better

ContextMatcher: Detector-Free Feature Matching with Cross-Modality Context

Improving Transformer-based Image Matching by Cascaded Capturing Spatially Informative Keypoints

FMRT: Learning Accurate Feature Matching with Reconciliatory Transformer

Local feature matching from detector-based to detector-free: a survey

A Robust Multisource Remote Sensing Image Matching Method Utilizing Attention and Feature Enhancement Against Noise Interference