Abstract:Image registration is the basis for joint utilization of multisource scene information. However, accurate automatic registration of multisource remote sensing images remains a challenging task, especially for optical and synthetic aperture radar (SAR) images. Due to the large geometric and intensity differences between images, many algorithms often fail to accurately register or even mismatch. In this article, we propose a novel coarse-to-fine method with stable high accuracy, which mainly consists of three steps. First, extract consistent features for robust coarse registration. Considering the differences in local gradient magnitudes between optical and SAR images, image intensity preprocessing is performed. Meanwhile, instead of the simple calculations via horizontal and vertical gradient operators, multidirectional gradient operators are defined in two scale spaces to obtain consistent local gradient information. Via multidirectional consistent gradients, highly repeatable keypoints are detected. On the multidirectional gradient maps, support regions with multiple scales are utilized to construct multiscale and multidirectional consistent cross-modal feature descriptors. Second, match the extracted features. Aiming to obtain a reliable alignment in coarse registration step, novel strategies are implemented for the first matching of a cascaded feature matching method. Sufficiently reliable initial matches are established via a new two-way matching strategy, and obvious outliers are removed by exploring the consistencies of both spatial scale and local neighborhood elements of the correct matches. Third, form distinctive pixelwise feature representations for accurate fine registration. In order to increase the distinctiveness of features to distinguish adjacent pixels, a new filter bank based on small receptive fields is defined. Fine features are constructed, appearing as thinner structures at the edges. Meanwhile, through multiscale and multidirectional convolutions, sufficient neighborhood information is mined. Therefore, more precise correspondences can be found to fine-tune the roughly corrected image pair. Overall, a combination method is proposed with a feature-based method and an area-based method for coarse registration and fine registration, respectively. On simulated and real image pairs, the above three steps and the two-stage framework are verified. Experimental results show that the proposed optical-to-SAR image registration method based on the designed multiscale, multidirectional consistent feature and multiscale, multidirectional fine feature (M2F2M) is superior to the current representative feature-based and area-based methods in robustness and accuracy.

Robust Optical and SAR Image Matching Using Attention-Enhanced Structural Features

Robust Matching for SAR and Optical Images Using Multiscale Convolutional Gradient Features

Shared contents alignment across multiple granularities for robust SAR-optical image matching

An Attention-Based Fusion for Handcrafted and Deep Feature to Improve Optical and SAR Image Matching

Multifeature Alignment and Matching Network for SAR and Optical Image Registration

Feature Matching and Position Matching Between Optical and SAR with Local Deep Feature Descriptor

Optical and SAR Image Matching Using Pixelwise Deep Dense Features

Multi-Feature Alignment and Matching Network for SAR and Optical Image Registration

Explore Better Network Framework for High-Resolution Optical and SAR Image Matching

SAR and Optical Image Registration Based on Deep Learning with Co-Attention Matching Module

SAR-Optical Image Matching by Integrating Siamese U-Net with FFT Correlation

SAR-optical Feature Matching: A Large-Scale Patch Dataset and a Deep Local Descriptor

MAP-Net: SAR and Optical Image Matching via Image-Based Convolutional Network With Attention Mechanism and Spatial Pyramid Aggregated Pooling

Detector-Free Feature Matching for Optical and SAR Images Based on a Two-Step Strategy

Synthetic aperture radar and optical image registration using local and global feature learning by modality-shared attention network

Scene-Level Matching Between Remote Sensing Optical and SAR Images

Fast and Robust Matching for Multimodal Remote Sensing Image Registration

LSV-ANet: Deep Learning on Local Structure Visualization for Feature Matching

Robust Coarse-to-Fine Registration Algorithm for Optical and SAR Images Based on Two Novel Multi-Scale and Multi-Directional Features

SFOC: A NOVEL MULTI-DIRECTIONAL AND MULTI-SCALE STRUCTURAL DESCRIPTOR FOR MULTIMODAL REMOTE SENSING IMAGE MATCHING

A Robust Multisource Remote Sensing Image Matching Method Utilizing Attention and Feature Enhancement Against Noise Interference