Abstract:Image registration is the basis for joint utilization of multisource scene information. However, accurate automatic registration of multisource remote sensing images remains a challenging task, especially for optical and synthetic aperture radar (SAR) images. Due to the large geometric and intensity differences between images, many algorithms often fail to accurately register or even mismatch. In this article, we propose a novel coarse-to-fine method with stable high accuracy, which mainly consists of three steps. First, extract consistent features for robust coarse registration. Considering the differences in local gradient magnitudes between optical and SAR images, image intensity preprocessing is performed. Meanwhile, instead of the simple calculations via horizontal and vertical gradient operators, multidirectional gradient operators are defined in two scale spaces to obtain consistent local gradient information. Via multidirectional consistent gradients, highly repeatable keypoints are detected. On the multidirectional gradient maps, support regions with multiple scales are utilized to construct multiscale and multidirectional consistent cross-modal feature descriptors. Second, match the extracted features. Aiming to obtain a reliable alignment in coarse registration step, novel strategies are implemented for the first matching of a cascaded feature matching method. Sufficiently reliable initial matches are established via a new two-way matching strategy, and obvious outliers are removed by exploring the consistencies of both spatial scale and local neighborhood elements of the correct matches. Third, form distinctive pixelwise feature representations for accurate fine registration. In order to increase the distinctiveness of features to distinguish adjacent pixels, a new filter bank based on small receptive fields is defined. Fine features are constructed, appearing as thinner structures at the edges. Meanwhile, through multiscale and multidirectional convolutions, sufficient neighborhood information is mined. Therefore, more precise correspondences can be found to fine-tune the roughly corrected image pair. Overall, a combination method is proposed with a feature-based method and an area-based method for coarse registration and fine registration, respectively. On simulated and real image pairs, the above three steps and the two-stage framework are verified. Experimental results show that the proposed optical-to-SAR image registration method based on the designed multiscale, multidirectional consistent feature and multiscale, multidirectional fine feature (M2F2M) is superior to the current representative feature-based and area-based methods in robustness and accuracy.

Intra- and Inter-Modal Graph Attention Network and Contrastive Learning for SAR and Optical Image Registration

Synthetic aperture radar and optical image registration using local and global feature learning by modality-shared attention network

Synthetic Aperture Radar and Optical Images Registration Based on Convolutional and Graph Neural Networks

Multifeature Alignment and Matching Network for SAR and Optical Image Registration

Multi-Feature Alignment and Matching Network for SAR and Optical Image Registration

A Global-to-Local Algorithm for High-Resolution Optical and SAR Image Registration.

SAR and Optical Image Registration Based on Deep Learning with Co-Attention Matching Module

A Novel Dual-Branch Global and Local Feature Extraction Network for SAR and Optical Image Registration

Deep Generative Matching Network for Optical and SAR Image Registration

Cross-Modal Information-Guided Network Using Contrastive Learning for Point Cloud Registration

Adaptive Registration for Optical and SAR Images with a Scale-Constrained Matching Method

Optical and SAR Image Registration Based on Pseudo-SAR Image Generation Strategy

Contrastive Learning-Based Dual Dynamic GCN for SAR Image Scene Classification.

K-Means Clustering Guided Generative Adversarial Networks for SAR-Optical Image Matching

Robust Coarse-to-Fine Registration Algorithm for Optical and SAR Images Based on Two Novel Multi-Scale and Multi-Directional Features

Multimodal Affine Registration for ICGA and MCSL Fundus Images of High Myopia

Self-Supervised Classification of SAR Images with Optical Image Assistance

A GAN Based Multi-Contrast Modalities Medical Image Registration Approach

Hybrid Cgan: Coupling Global and Local Features for SAR-to-Optical Image Translation

MGAN: Attempting a Multimodal Graph Attention Network for Remote Sensing Cross-Modal Text-Image Retrieval