Abstract:The performance of the different combinations of feature extractor, descriptor, and similarity measure varies between individual datasets. The combination of SIFT, ContextDesc, and NNDR, and also the original SIFT, achieve the best results when using more comprehensive indicators on all the datasets. For extractor, the learning based Sub‐SuperPoint is second only to SIFT; for descriptor, learning based ContextDesc is as effective as the SIFT descriptor; for matcher, learning based SuperGlue is not as stable as NNDR. Image matching plays an important role in photogrammetry, computer vision and remote sensing. Modern deep learning‐based methods have been proposed for image matching; however, whether they will surpass and take the place of the conventional handcrafted methods in the remote sensing field still remains unclear. A comprehensive evaluation on stereo remote sensing images is also lacking. This paper comprehensively evaluates the performance of conventional and deep learning‐based image‐matching methods by dividing the matching process into feature point extraction, description and similarity measure on various datasets, including images captured from close‐range indoor and outdoor scenarios, unmanned aerial vehicles (UAVs) and satellite platforms. Different combinations of the three steps are evaluated. The experimental results reveal that, first, the performance of the different combinations varies between individual datasets, and it is difficult to determine the best combination. Second, by using more comprehensive indicators on all of the datasets, that is, the average rank and absolute rank, the combination of scale‐invariant feature transform (SIFT), ContextDesc and the nearest neighbour distance ratio (NNDR), and also the original SIFT, achieve the best results, and are recommended for use in remote sensing. Third, the deep learning‐based Sub‐SuperPoint extractor obtains a good performance, and is second only to SIFT. The learning based ContextDesc descriptor is as effective as the SIFT descriptor, and the learning based SuperGlue matcher is not as stable as NNDR, but leads to a few top‐performing combinations. Finally, the handcrafted methods are generally faster than the deep learning‐based methods, but the efficiency of the latter is acceptable. We conclude that although a full deep learning‐based method/combination has not yet beaten the conventional methods, there is still much room for improvement with the deep learning‐based methods because large‐scale aerial and satellite training datasets remain to be constructed, and specific methods for remote sensing images remain to be developed. The performance of the different combinations of feature extractor, descriptor and similarity measure varies between individual datasets. The combination of SIFT, ContextDesc and NNDR, and also the original SIFT, achieve the best results when using more comprehensive indicators on all the datasets. For extractor, the learning based Sub‐SuperPoint is second only to SIFT; for descriptor, learning‐based ContextDesc is as effective as the SIFT descriptor; and for matcher, learning‐based SuperGlue is not as stable as NNDR.

A Light-weight Transformer-based Self-supervised Matching Network for Heterogeneous Images

LTFormer: A Light-Weight Transformer-Based Self-Supervised Matching Network for Heterogeneous Remote Sensing Images

A Coarse-to-Fine Feature Match Network Using Transformers for Remote Sensing Image Registration.

Unsupervised Multi-Spectral Image Feature Point Matching under Different Lighting Scenes

Self-Supervised Keypoint Detection and Cross-Fusion Matching Networks for Multimodal Remote Sensing Image Registration

LM-Net: A Lightweight Matching Network for Remote Sensing Image Matching and Registration

Learning Enriched Feature Descriptor for Image Matching and Visual Measurement

SAR–Optical Image Matching Using Self-Supervised Detection and a Transformer–CNN-Based Network

A Novel Local Pettern Based Self-Similarity Descriptor for Multisource Remote Sensing Image Registration.

LSV-ANet: Deep Learning on Local Structure Visualization for Feature Matching

Multimodal Remote Sensing Image Matching via Learning Features and Attention Mechanism

A Novel Neural Network for Remote Sensing Image Matching

Image Matching and Localization Based on Fusion of Handcrafted and Deep Features

A Robust Multisource Remote Sensing Image Matching Method Utilizing Attention and Feature Enhancement Against Noise Interference

Deep Siamese Multi-level Feature Network for VIS and NIR Image Matching

MatchFormer: Interleaving Attention in Transformers for Feature Matching

Remote Sensing Multimodal Image Matching Based on Structure Feature and Learnable Matching Network

STFNet: Self-Supervised Transformer for Infrared and Visible Image Fusion

An evaluation of conventional and deep learning‐based image‐matching methods on diverse datasets

DeepMatcher: A Deep Transformer-based Network for Robust and Accurate Local Feature Matching

Optical and SAR Image Matching Using Pixelwise Deep Dense Features