Abstract:Introduction: Optical and SAR image matching is one of the fields within multi-sensor imaging and fusion. It is crucial for various applications such as disaster response, environmental monitoring, and urban planning, as it enables comprehensive and accurate analysis by combining the visual information of optical images with the penetrating capability of SAR images. However, the differences in imaging mechanisms between optical and SAR images result in significant nonlinear radiation distortion. Especially for SAR images, which are affected by speckle noises, resulting in low resolution and blurry edge structures, making optical and SAR image matching difficult and challenging. The key to successful matching lies in reducing modal differences and extracting similarity information from the images. Method: In light of this, we propose a structure similarity virtual map generation network (SVGNet) to address the task of optical and SAR image matching. The core innovation of this paper is that we take inspiration from the concept of image generation, to handle the predicament of image matching between different modalities. Firstly, we introduce the Attention U-Net as a generator to decouple and characterize optical images. And then, SAR images are consistently converted into optical images with similar textures and structures. At the same time, using the structural similarity (SSIM) to constrain structural spatial information to improve the quality of generated images. Secondly, a conditional generative adversarial network is employed to further guide the image generation process. By combining synthesized SAR images and their corresponding optical images in a dual channel, we can enhance prior information. This combined data is then fed into the discriminator to determine whether the images are true or false, guiding the generator to optimize feature learning. Finally, we employ least squares loss (LSGAN) to stabilize the training of the generative adversarial network. Results and Discussion: Experiments have demonstrated that the SVGNet proposed in this paper is capable of effectively reducing modal differences, and it increases the matching success rate. Compared to direct image matching, using image generation ideas results in a matching accuracy improvement of more than twice.

IGS-Net: Seeking Good Correspondences via Interactive Generative Structure Learning

CSR-Net++: Rethinking Context Structure Representation Learning for Feature Matching

StateNet: Deep State Learning for Robust Feature Matching of Remote Sensing Images

Structure similarity virtual map generation network for optical and SAR image matching

CSR-Net: Learning Adaptive Context Structure Representation for Robust Feature Correspondence

LSV-ANet: Deep Learning on Local Structure Visualization for Feature Matching

Intra- and Intersource Interactive Representation Learning Network for Remote Sensing Images Classification

Robust Feature Matching via Hierarchical Local Structure Visualization

Remote Sensing Multimodal Image Matching Based on Structure Feature and Learnable Matching Network

A Novel Neural Network for Remote Sensing Image Matching

A Hierarchical Consensus Attention Network for Feature Matching of Remote Sensing Images

VLSG-SANet: A feature matching algorithm for remote sensing image registration

Explore Better Network Framework for High-Resolution Optical and SAR Image Matching

Superpixel-Based Long-Range Dependent Network for High-Resolution Remote-Sensing Image Classification

Robust Optical and SAR Image Matching Using Attention-Enhanced Structural Features

SCSA-Net: Presentation of two-view reliable correspondence learning via spatial-channel self-attention

GLoCNet: Robust Feature Matching With Global–Local Consistency Network for Remote Sensing Image Registration

A Dual-Generator Translation Network Fusing Texture and Structure Features for SAR and Optical Image Matching

Multimodal Remote Sensing Image Matching via Learning Features and Attention Mechanism

Fine-Grained Information Supplementation and Value-Guided Learning for Remote Sensing Image-Text Retrieval

PG-Net: Progressive Guidance Network via Robust Contextual Embedding for Efficient Point Cloud Registration