Abstract:The complementarity of synthetic aperture radar (SAR) and optical images allows remote sensing observations to "see" unprecedented discoveries. Image matching plays a fundamental role in the fusion and application of SAR and optical images. However, both the geometric imaging pattern and the physical radiation mechanism of these two sensors are significantly different, so that the images show complex geometric distortion and nonlinear radiation differences. This phenomenon brings great challenges to image matching, which neither the handcrafted descriptors nor the deep learning-based methods have adequately addressed. In this article, a novel image-based matching method for SAR to optical images via an image-based convolutional network with spatial pyramid aggregated pooling (SPAP) and an attention mechanism is proposed, namely MAP-Net. The original image is embedded through the convolutional neural network to generate the feature map. Through the information extraction and abstraction of the original imagery, the embedded features containing the high-level semantic information are more robust to the geometric distortion and radiation variation among the different modal images, which is beneficial to the matching of cross-modal images. The adoption of the SPAP module makes the network more capable of integrating global and local contextual information. The attention block weights the dense features generated from the network to extract the key features that are invariant, distinguishable, repeatable, and suitable for the image matching task. In the experiments, five sets of multisource and multiresolution SAR and optical images with wide and varied ground coverage were used to evaluate the accuracy of MAP-Net, compared to both handcrafted and deep learning-based methods. The experimental results show that the MAP-Net method is superior to the current state-of-the-art image matching methods for SAR to optical images.

SAR–Optical Image Matching Using Self-Supervised Detection and a Transformer–CNN-Based Network

Optical and SAR Image Matching Using Pixelwise Deep Dense Features

Multilevel Attention Siamese Network for Keypoint Detection in Optical and SAR Images

Unsupervised SAR and Optical Image Matching Using Siamese Domain Adaptation

A Novel Local Pettern Based Self-Similarity Descriptor for Multisource Remote Sensing Image Registration.

Robust Local Feature Descriptor for Multisource Remote Sensing Image Registration

Self-Supervised Classification of SAR Images with Optical Image Assistance

Self-Supervised Keypoint Detection and Cross-Fusion Matching Networks for Multimodal Remote Sensing Image Registration

Explore Better Network Framework for High-Resolution Optical and SAR Image Matching

A Light-weight Transformer-based Self-supervised Matching Network for Heterogeneous Images

Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN

A lightweight deep convolutional network with inverted residuals for matching optical and SAR images

Synthetic Aperture Radar and Optical Images Registration Based on Convolutional and Graph Neural Networks

MAP-Net: SAR and Optical Image Matching via Image-Based Convolutional Network With Attention Mechanism and Spatial Pyramid Aggregated Pooling

Self-Distillation Feature Learning Network for Optical and SAR Image Registration

Robust Optical and SAR Image Matching Using Attention-Enhanced Structural Features

Classification for SAR Scene Matching Areas Based on Convolutional Neural Networks.

Detector-Free Feature Matching for Optical and SAR Images Based on a Two-Step Strategy

SD-CapsNet: A Siamese Dense Capsule Network for SAR Image Registration with Complex Scenes

Robust Matching for SAR and Optical Images Using Multiscale Convolutional Gradient Features

SAR-CDSS: A Semi-Supervised Cross-Domain Object Detection from Optical to SAR Domain