Abstract:The fusion of image data from multiple sensors is crucial for many applications. However, there are significant nonlinear intensity deformations between images from different kinds of sensors, leading to matching failure. To address this need, this paper proposes an effective coarse-to-fine matching method for multimodal remote sensing images (3MRS). In the coarse matching stage, feature points are first detected on a maximum moment map calculated with a phase congruency model. Then, feature description is conducted using an index map constructed by finding the index of the maximum value in all orientations of convolved images obtained using a set of log-Gabor filters. At last, several matches are built through image matching and outlier removal, which can be used to estimate a reliable affine transformation model between the images. In the stage of fine matching, we develop a novel template matching method based on the log-Gabor convolution image sequence and match the template features with a 3D phase correlation matching strategy, given that the initial correspondences are achieved with the estimated transformation. Results show that compared with SIFT, and three state-of-the-art methods designed for multimodal image matching, PSO-SIFT, HAPCG, and RIFT, only 3MRS successfully matched all six types of multimodal remote sensing image pairs: optical–optical, optical–infrared, optical–depth, optical–map, optical–SAR, and day–night, with each including ten different image pairs. On average, the number of correct matches (NCM) of 3MRS was 164.47, 123.91, 4.88, and 4.33 times that of SIFT, PSO-SIFT, HAPCG, and RIFT for the successfully matched image pairs of each method. In terms of accuracy, the root-mean-square error of correct matches for 3MRS, SIFT, PSO-SIFT, HAPCG, and RIFT are 1.47, 1.98, 1.79, 2.83, and 2.45 pixels, respectively, revealing that 3MRS got the highest accuracy. Even though the total running time of 3MRS was the longest, the efficiency for obtaining one correct match is the highest considering the most significant number of matches. The source code of 3MRS and the experimental datasets and detailed results are publicly available.

A radiometric and rotational invariant feature descriptor for multimodal remote sensing image matching using the monogenic signal

Robust Local Feature Descriptor for Multisource Remote Sensing Image Registration

Dynamic Convolution Covariance Network Using Multi-Scale Feature Fusion for Remote Sensing Scene Image Classification

Multi-Modal Remote Sensing Image Matching Considering Co-Occurrence Filter

Modality-Free Feature Detector and Descriptor for Multimodal Remote Sensing Image Registration

A Novel Robust Feature Descriptor for Multi-Source Remote Sensing Image Registration

A Novel Local Pettern Based Self-Similarity Descriptor for Multisource Remote Sensing Image Registration.

Robust Multimodal Remote Sensing Image Matching Based on Enhanced Oriented Self-Similarity Descriptor

Srifd: A Shift Rotation Invariant Feature Descriptor for Multi-Sensor Image Matching

Multimodal image matching: A scale-invariant algorithm and an open dataset

Multimodal Remote Sensing Image Matching via Learning Features and Attention Mechanism

A novel multimodal remote-sensing image registration algorithm using phase symmetry and rank-based local self-similarity

3MRS: An Effective Coarse-to-Fine Matching Method for Multimodal Remote Sensing Imagery

REMM:Rotation-Equivariant Framework for End-to-End Multimodal Image Matching

Fast and Robust Matching for Multimodal Remote Sensing Image Registration

GRiD: Guided Refinement for Detector-Free Multimodal Image Matching

RI-LPOH: Rotation-Invariant Local Phase Orientation Histogram for Multi-Modal Image Matching

A High Precision Feature Matching Method Based On Geometrical Outlier Removal For Remote Sensing Image Registration

Multimodal Remote Sensing Image Registration Based on Adaptive Spectrum Congruency

RIFT: Multi-Modal Image Matching Based on Radiation-Variation Insensitive Feature Transform

A Robust Multisource Remote Sensing Image Matching Method Utilizing Attention and Feature Enhancement Against Noise Interference