Abstract:The complementarity of synthetic aperture radar (SAR) and optical images allows remote sensing observations to "see" unprecedented discoveries. Image matching plays a fundamental role in the fusion and application of SAR and optical images. However, both the geometric imaging pattern and the physical radiation mechanism of these two sensors are significantly different, so that the images show complex geometric distortion and nonlinear radiation differences. This phenomenon brings great challenges to image matching, which neither the handcrafted descriptors nor the deep learning-based methods have adequately addressed. In this article, a novel image-based matching method for SAR to optical images via an image-based convolutional network with spatial pyramid aggregated pooling (SPAP) and an attention mechanism is proposed, namely MAP-Net. The original image is embedded through the convolutional neural network to generate the feature map. Through the information extraction and abstraction of the original imagery, the embedded features containing the high-level semantic information are more robust to the geometric distortion and radiation variation among the different modal images, which is beneficial to the matching of cross-modal images. The adoption of the SPAP module makes the network more capable of integrating global and local contextual information. The attention block weights the dense features generated from the network to extract the key features that are invariant, distinguishable, repeatable, and suitable for the image matching task. In the experiments, five sets of multisource and multiresolution SAR and optical images with wide and varied ground coverage were used to evaluate the accuracy of MAP-Net, compared to both handcrafted and deep learning-based methods. The experimental results show that the MAP-Net method is superior to the current state-of-the-art image matching methods for SAR to optical images.

SAR-Optical Image Matching by Integrating Siamese U-Net with FFT Correlation

Optical and SAR Image Matching Using Pixelwise Deep Dense Features

Feature Matching and Position Matching Between Optical and SAR with Local Deep Feature Descriptor

SAR-optical Feature Matching: A Large-Scale Patch Dataset and a Deep Local Descriptor

Explore Better Network Framework for High-Resolution Optical and SAR Image Matching

Unsupervised SAR and Optical Image Matching Using Siamese Domain Adaptation

SAR and Optical Image Registration Based on Deep Learning with Co-Attention Matching Module

Application of Deep Learning for Registration Between SAR and Optical Images

MAP-Net: SAR and Optical Image Matching via Image-Based Convolutional Network With Attention Mechanism and Spatial Pyramid Aggregated Pooling

SAR–Optical Image Matching Using Self-Supervised Detection and a Transformer–CNN-Based Network

Robust Optical and SAR Image Matching Using Attention-Enhanced Structural Features

Shared contents alignment across multiple granularities for robust SAR-optical image matching

Scene-Level Matching Between Remote Sensing Optical and SAR Images

Cosine Similarity Template Matching Networks for Optical and SAR Image Registration

A lightweight deep convolutional network with inverted residuals for matching optical and SAR images

Synthetic Aperture Radar and Optical Images Registration Based on Convolutional and Graph Neural Networks

Synthetic aperture radar and optical image registration using local and global feature learning by modality-shared attention network

Self-Distillation Feature Learning Network for Optical and SAR Image Registration

Multifeature Alignment and Matching Network for SAR and Optical Image Registration

Him-Net - A New Neural Network Approach for SAR and Optical Image Template Matching1.

Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN