Abstract:The complementarity of synthetic aperture radar (SAR) and optical images allows remote sensing observations to "see" unprecedented discoveries. Image matching plays a fundamental role in the fusion and application of SAR and optical images. However, both the geometric imaging pattern and the physical radiation mechanism of these two sensors are significantly different, so that the images show complex geometric distortion and nonlinear radiation differences. This phenomenon brings great challenges to image matching, which neither the handcrafted descriptors nor the deep learning-based methods have adequately addressed. In this article, a novel image-based matching method for SAR to optical images via an image-based convolutional network with spatial pyramid aggregated pooling (SPAP) and an attention mechanism is proposed, namely MAP-Net. The original image is embedded through the convolutional neural network to generate the feature map. Through the information extraction and abstraction of the original imagery, the embedded features containing the high-level semantic information are more robust to the geometric distortion and radiation variation among the different modal images, which is beneficial to the matching of cross-modal images. The adoption of the SPAP module makes the network more capable of integrating global and local contextual information. The attention block weights the dense features generated from the network to extract the key features that are invariant, distinguishable, repeatable, and suitable for the image matching task. In the experiments, five sets of multisource and multiresolution SAR and optical images with wide and varied ground coverage were used to evaluate the accuracy of MAP-Net, compared to both handcrafted and deep learning-based methods. The experimental results show that the MAP-Net method is superior to the current state-of-the-art image matching methods for SAR to optical images.

Synthetic Aperture Radar Scene Classification Using Multiview Cross Correlation Attention Network

Attention in Attention for Hyperspectral with High Spatial Resolution (H) Image Classification

Parallel-Connected Residual Channel Attention Network for Remote Sensing Image Super-Resolution

Synthetic aperture radar remote sensing scene classification based on fuzzy co-occurrence networks

MS2CANet: Multiscale Spatial–Spectral Cross-Modal Attention Network for Hyperspectral Image and LiDAR Classification

Cross-domain attention network for hyperspectral image classification

Classification for SAR Scene Matching Areas Based on Convolutional Neural Networks.

ASANet: Asymmetric Semantic Aligning Network for RGB and SAR image land cover classification

Channel-Attention-Based DenseNet Network for Remote Sensing Image Scene Classification

High-Resolution SAR Image Classification Using Multi-Scale Deep Feature Fusion and Covariance Pooling Manifold Network

Multi-Scale Convolutional Neural Network for SAR Image Semantic Segmentation

Relation-Attention Networks for Remote Sensing Scene Classification

MPT-SFANet: Multi-Order Pooling Transformer-Based Semantic Feature Aggregation Network for SAR Image Classification

SAR image classification using deep features and neighborhood heterogeneity confidence refined spatial constraints

A multi-scale multi-channel CNN introducing a channel-spatial attention mechanism hyperspectral remote sensing image classification method

High Resolution SAR Image Classification Using Global-Local Network Structure Based on Vision Transformer and CNN

A Feature Fusion-Net Using Deep Spatial Context Encoder and Nonstationary Joint Statistical Model for High-Resolution SAR Image Classification

MAP-Net: SAR and Optical Image Matching via Image-Based Convolutional Network With Attention Mechanism and Spatial Pyramid Aggregated Pooling

Multimodal Bilinear Fusion Network with Second-Order Attention-Based Channel Selection for Land Cover Classification

Deep neural network based on attention and feature complementary fusion for synthetic aperture radar image classification with small samples