Abstract:Remote sensing image matching is the basis upon which to obtain integrated observations and complementary information representation of the same scene from multiple source sensors, which is a prerequisite for remote sensing tasks such as remote sensing image fusion and change detection. However, the intricate geometric and radiometric differences between the multimodal images render the registration quite challenging. Although multimodal remote sensing image matching methods have been developed in recent decades, most classical and deep learning based techniques cannot effectively extract high repeatable keypoints and discriminative descriptors for multimodal images. Therefore, we propose a two-step "detection + matching" framework in this paper, where each step consists of a deep neural network. A self-supervised detection network is first designed to generate similar keypoint feature maps between multimodal images, which is used to detect highly repeatable keypoints. We then propose a cross-fusion matching network, which aims to exploit global optimization and fusion information for cross-modal feature descriptors and matching. The experiments show that the proposed method has superior feature detection and matching performance compared with current state-of-the-art methods. Specifically, the keypoint repetition rate of the detection network and the NN mAP of the matching network are 0.435 and 0.712 on test datasets, respectively. The proposed whole pipeline framework is evaluated, which achieves an average M.S. and RMSE of 0.298 and 3.41, respectively. This provides a novel solution for the joint use of multimodal remote sensing images for observation and localization.

Multimodal Image Registration by Iteratively Searching Keypoint Correspondences.

Incorporating Global Information in Feature-Based Multimodal Image Registration

Image Registration Method for Multimodal Images

Multimodal Image Registration with Line Segments by Selective Search.

Multi-source Remote Sensing Image Registration Based on Sift and Optimization of Local Self-Similarity Mutual Information

Multi-sensor Images Registration Based on SIFT and Extended Phase Correlation

Multimodal Remote Sensing Image Registration Based on Image Transfer and Local Features

Multimodal image registration based on orientation-moment

Self-Supervised Keypoint Detection and Cross-Fusion Matching Networks for Multimodal Remote Sensing Image Registration

Automatic Remote Sensing Image Registration Algorithm Based on SIFT

Robust Multimodal Remote Sensing Image Registration Based on Local Statistical Frequency Information

Multi-Modal Image Registration Based on Local Self-Similarity and Bidirectional Matching

A Two-Step Method for Remote Sensing Images Registration Based on Local and Global Constraints

Multi-sensor Image Registration by Combining Local Self-Similarity Matching and Mutual Information

Study and Implementation of Parallel Registration Algorithm for Multimodality Remote-Sensing Images Based on Mutual Information

A Fast and Robust Matching System for Multimodal Remote Sensing Image Registration

Establishing Keypoint Matches on Multimodal Images With Bootstrap Strategy and Global Information.

A Point-Based Algorithm for Multi-Spectral Image Registration Using Mutual Information of Regions

A New Registration Algorithm for Multimodal Remote Sensing Images.

Multi-sensor Optical Remote Sensing Image Registration Based on Line-Point Invariant.

Reliable and Fast Mapping of Keypoints on Large-Size Remote Sensing Images by Use of Multiresolution and Global Information