Abstract:Feature based image matching has been a research focus in photogrammetry and computer vision for decades, as it is the basis for many applications where multi-view geometry is needed. A typical feature based image matching algorithm contains five steps: feature detection, affine shape estimation, orientation assignment, description and descriptor matching. This paper contains innovative work in different steps of feature matching based on convolutional neural networks (CNN). For the affine shape estimation and orientation assignment, the main contribution of this paper is twofold. First, we define a canonical shape and orientation for each feature. As a consequence, instead of the usual Siamese CNN, only single branch CNNs needs to be employed to learn the affine shape and orientation parameters, which turns the related tasks from supervised to self supervised learning problems, removing the need for known matching relationships between features. Second, the affine shape and orientation are solved simultaneously. To the best of our knowledge, this is the first time these two modules are reported to have been successfully trained together. In addition, for the descriptor learning part, a new weak match finder is suggested to better explore the intra-variance of the appearance of matched features. For any input feature patch, a transformed patch that lies far from the input feature patch in descriptor space is defined as a weak match feature. A weak match finder network is proposed to actively find these weak match features; they are subsequently used in the standard descriptor learning framework. The proposed modules are integrated into an inference pipeline to form the proposed feature matching algorithm. The algorithm is evaluated on standard benchmarks and is used to solve for the parameters of image orientation of aerial oblique images. It is shown that deep learning feature based image matching leads to more registered images, more reconstructed 3D points and a more stable block geometry than conventional methods. The code is available at https://github.com/Childhoo/Chen_Matcher.git.

Aerial image matching based on dynamic convolution and adaptive neighborhood consensus network

Real-time and High Precision Feature Matching Between Blur Aerial Images

Selective Context Network with Neighbourhood Consensus for Aerial Image Registration

Cross-view Matching Neural Network for Remote Sensing Images.

Aerial Scene Classification Via Multilevel Fusion Based on Deep Convolutional Neural Networks.

UNSUPERVISED MULTI-CONSTRAINT DEEP NEURAL NETWORK FOR DENSE IMAGE MATCHING

An Ensemble Method for Accurate Image Matching

Extended Neighborhood Consensus With Affine Correspondence for Outlier Filtering in Feature Matching

An Adaptive Remote Sensing Image-Matching Network Based on Cross Attention and Deformable Convolution

A ConvNeXt-Based Network for Aerial-view Geo-localization in Multiple Environment

Efficient Dynamic Correspondence Network

Airborne Image Matching Based on Fine-Grained Features and Mutual Attention

2Chadcnn: A Template Matching Network for Season-Changing UAV Aerial Images and Satellite Imagery

Multiview Image Matching of Optical Satellite and UAV Based on a Joint Description Neural Network

A Two-Stream Symmetric Network with Bidirectional Ensemble for Aerial Image Matching

CSR-Net++: Rethinking Context Structure Representation Learning for Feature Matching

A lightweight deep convolutional network with inverted residuals for matching optical and SAR images

CNN-Based Dense Image Matching for Aerial Remote Sensing Images

Deep learning feature representation for image matching under large viewpoint and viewing direction change

A Hierarchical Consensus Attention Network for Feature Matching of Remote Sensing Images

Deep Matching Network Based Image Auxiliary Localization System For Unmanned Aerial Vehicle