Abstract:Introduction: Optical and SAR image matching is one of the fields within multi-sensor imaging and fusion. It is crucial for various applications such as disaster response, environmental monitoring, and urban planning, as it enables comprehensive and accurate analysis by combining the visual information of optical images with the penetrating capability of SAR images. However, the differences in imaging mechanisms between optical and SAR images result in significant nonlinear radiation distortion. Especially for SAR images, which are affected by speckle noises, resulting in low resolution and blurry edge structures, making optical and SAR image matching difficult and challenging. The key to successful matching lies in reducing modal differences and extracting similarity information from the images. Method: In light of this, we propose a structure similarity virtual map generation network (SVGNet) to address the task of optical and SAR image matching. The core innovation of this paper is that we take inspiration from the concept of image generation, to handle the predicament of image matching between different modalities. Firstly, we introduce the Attention U-Net as a generator to decouple and characterize optical images. And then, SAR images are consistently converted into optical images with similar textures and structures. At the same time, using the structural similarity (SSIM) to constrain structural spatial information to improve the quality of generated images. Secondly, a conditional generative adversarial network is employed to further guide the image generation process. By combining synthesized SAR images and their corresponding optical images in a dual channel, we can enhance prior information. This combined data is then fed into the discriminator to determine whether the images are true or false, guiding the generator to optimize feature learning. Finally, we employ least squares loss (LSGAN) to stabilize the training of the generative adversarial network. Results and Discussion: Experiments have demonstrated that the SVGNet proposed in this paper is capable of effectively reducing modal differences, and it increases the matching success rate. Compared to direct image matching, using image generation ideas results in a matching accuracy improvement of more than twice.

MV-MOE: A Visual Mixture-of-Experts Model for Optical-SAR Image Matching

Robust Optical and SAR Image Matching Using Attention-Enhanced Structural Features

Multifeature Alignment and Matching Network for SAR and Optical Image Registration

Multimodal Remote Sensing Image Matching via Learning Features and Attention Mechanism

Optical and SAR Image Registration Based on Multi-Scale Orientated Map of Phase Congruency

Structure similarity virtual map generation network for optical and SAR image matching

MAP-Net: SAR and Optical Image Matching via Image-Based Convolutional Network With Attention Mechanism and Spatial Pyramid Aggregated Pooling

Fast and Robust Matching for Multimodal Remote Sensing Image Registration

Remote Sensing Multimodal Image Matching Based on Structure Feature and Learnable Matching Network

SAR–Optical Image Matching Using Self-Supervised Detection and a Transformer–CNN-Based Network

3MOS: Multi-sources, Multi-resolutions, and Multi-scenes dataset for Optical-SAR image matching

Explore Better Network Framework for High-Resolution Optical and SAR Image Matching

Optical and SAR Image Matching Using Pixelwise Deep Dense Features

SAR and Optical Image Registration Based on Deep Learning with Co-Attention Matching Module

A Robust Strategy for Large-Size Optical and SAR Image Registration

Automatic Registration of Optical and SAR Images Via Improved Phase Congruency Model.

Detector-Free Feature Matching for Optical and SAR Images Based on a Two-Step Strategy

Robust Multimodal Remote Sensing Image Matching Based on Enhanced Oriented Self-Similarity Descriptor

Retrieval Across Optical and SAR Images with Deep Neural Network.

Multiview Image Matching of Optical Satellite and UAV Based on a Joint Description Neural Network

Shared contents alignment across multiple granularities for robust SAR-optical image matching