Abstract:Introduction: Optical and SAR image matching is one of the fields within multi-sensor imaging and fusion. It is crucial for various applications such as disaster response, environmental monitoring, and urban planning, as it enables comprehensive and accurate analysis by combining the visual information of optical images with the penetrating capability of SAR images. However, the differences in imaging mechanisms between optical and SAR images result in significant nonlinear radiation distortion. Especially for SAR images, which are affected by speckle noises, resulting in low resolution and blurry edge structures, making optical and SAR image matching difficult and challenging. The key to successful matching lies in reducing modal differences and extracting similarity information from the images. Method: In light of this, we propose a structure similarity virtual map generation network (SVGNet) to address the task of optical and SAR image matching. The core innovation of this paper is that we take inspiration from the concept of image generation, to handle the predicament of image matching between different modalities. Firstly, we introduce the Attention U-Net as a generator to decouple and characterize optical images. And then, SAR images are consistently converted into optical images with similar textures and structures. At the same time, using the structural similarity (SSIM) to constrain structural spatial information to improve the quality of generated images. Secondly, a conditional generative adversarial network is employed to further guide the image generation process. By combining synthesized SAR images and their corresponding optical images in a dual channel, we can enhance prior information. This combined data is then fed into the discriminator to determine whether the images are true or false, guiding the generator to optimize feature learning. Finally, we employ least squares loss (LSGAN) to stabilize the training of the generative adversarial network. Results and Discussion: Experiments have demonstrated that the SVGNet proposed in this paper is capable of effectively reducing modal differences, and it increases the matching success rate. Compared to direct image matching, using image generation ideas results in a matching accuracy improvement of more than twice.

Cosine Similarity Template Matching Networks for Optical and SAR Image Registration

Him-Net - A New Neural Network Approach for SAR and Optical Image Template Matching1.

SAR and Optical Image Matching Eased on Phase Congruency and Template Matching

SAR-Optical Image Matching by Integrating Siamese U-Net with FFT Correlation

Optical and SAR Image Matching Using Pixelwise Deep Dense Features

Shared contents alignment across multiple granularities for robust SAR-optical image matching

Feature Matching and Position Matching Between Optical and SAR with Local Deep Feature Descriptor

A Novel Rotation and Scale Equivariant Network for Optical-SAR Image Matching

A Robust Strategy for Large-Size Optical and SAR Image Registration

Template Coarse Matching of SAR and Optical Images Based on Wavelet Subbands and Hausdorff Distance

Synthetic Aperture Radar and Optical Images Registration Based on Convolutional and Graph Neural Networks

Explore Better Network Framework for High-Resolution Optical and SAR Image Matching

Multifeature Alignment and Matching Network for SAR and Optical Image Registration

Rank-Based Local Self-Similarity Descriptor for Optical-to-SAR Image Matching

MAP-Net: SAR and Optical Image Matching via Image-Based Convolutional Network With Attention Mechanism and Spatial Pyramid Aggregated Pooling

Remote Sensing Multimodal Image Matching Based on Structure Feature and Learnable Matching Network

Scene-Level Matching Between Remote Sensing Optical and SAR Images

Structure similarity virtual map generation network for optical and SAR image matching

Automatic Registration of Optical and SAR Images Via Improved Phase Congruency Model.

Robust Optical and SAR Image Matching Using Attention-Enhanced Structural Features

Fast and Robust Matching for Multimodal Remote Sensing Image Registration