Abstract:Feature based image matching has been a research focus in photogrammetry and computer vision for decades, as it is the basis for many applications where multi-view geometry is needed. A typical feature based image matching algorithm contains five steps: feature detection, affine shape estimation, orientation assignment, description and descriptor matching. This paper contains innovative work in different steps of feature matching based on convolutional neural networks (CNN). For the affine shape estimation and orientation assignment, the main contribution of this paper is twofold. First, we define a canonical shape and orientation for each feature. As a consequence, instead of the usual Siamese CNN, only single branch CNNs needs to be employed to learn the affine shape and orientation parameters, which turns the related tasks from supervised to self supervised learning problems, removing the need for known matching relationships between features. Second, the affine shape and orientation are solved simultaneously. To the best of our knowledge, this is the first time these two modules are reported to have been successfully trained together. In addition, for the descriptor learning part, a new weak match finder is suggested to better explore the intra-variance of the appearance of matched features. For any input feature patch, a transformed patch that lies far from the input feature patch in descriptor space is defined as a weak match feature. A weak match finder network is proposed to actively find these weak match features; they are subsequently used in the standard descriptor learning framework. The proposed modules are integrated into an inference pipeline to form the proposed feature matching algorithm. The algorithm is evaluated on standard benchmarks and is used to solve for the parameters of image orientation of aerial oblique images. It is shown that deep learning feature based image matching leads to more registered images, more reconstructed 3D points and a more stable block geometry than conventional methods. The code is available at https://github.com/Childhoo/Chen_Matcher.git.

Image Matching with Large Viewing Angle Based on Local Features

A computationally efficient affine-invariant feature for image matching with wide viewing angles

Leveraging Local Planar Motion Property for Robust Visual Matching and Localization.

Multi-source Remote Sensing Image Registration Based on Sift and Optimization of Local Self-Similarity Mutual Information

Robust Affine Invariant Feature Extraction for Image Matching

Feature Point Detection and Matching of Wide Baseline Image Based on Scale Space Theory and Guided Matching Algorithm

An Image Matching Algorithm Based on Improved Rank Transform

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning.

Image Matching for Non-Planar Scenes under Wide Baseline Views

Ensemble Learning with Advanced Fast Image Filtering Features for Semi-Global Matching

Efficient Covisibility-based Image Matching for Large-Scale SfM

Robust Local Feature Descriptor for Multisource Remote Sensing Image Registration

A robust deformed image matching method for multi-source image matching

A Novel Algorithm for View and Illumination Invariant Image Matching

Image matching based on a local invariant descriptor

Affine-invariant SIFT Descriptor with Global Context

Fine-search for image copy detection based on local affine-invariant descriptor and spatial dependent matching

Mutual information based non-parametric transform stereo matching algorithm

Local Invariant Descriptor for Image Matching

Deep learning feature representation for image matching under large viewpoint and viewing direction change

Image Matching by Multiscale Oriented Corner Correlation