Abstract:Introduction: Optical and SAR image matching is one of the fields within multi-sensor imaging and fusion. It is crucial for various applications such as disaster response, environmental monitoring, and urban planning, as it enables comprehensive and accurate analysis by combining the visual information of optical images with the penetrating capability of SAR images. However, the differences in imaging mechanisms between optical and SAR images result in significant nonlinear radiation distortion. Especially for SAR images, which are affected by speckle noises, resulting in low resolution and blurry edge structures, making optical and SAR image matching difficult and challenging. The key to successful matching lies in reducing modal differences and extracting similarity information from the images. Method: In light of this, we propose a structure similarity virtual map generation network (SVGNet) to address the task of optical and SAR image matching. The core innovation of this paper is that we take inspiration from the concept of image generation, to handle the predicament of image matching between different modalities. Firstly, we introduce the Attention U-Net as a generator to decouple and characterize optical images. And then, SAR images are consistently converted into optical images with similar textures and structures. At the same time, using the structural similarity (SSIM) to constrain structural spatial information to improve the quality of generated images. Secondly, a conditional generative adversarial network is employed to further guide the image generation process. By combining synthesized SAR images and their corresponding optical images in a dual channel, we can enhance prior information. This combined data is then fed into the discriminator to determine whether the images are true or false, guiding the generator to optimize feature learning. Finally, we employ least squares loss (LSGAN) to stabilize the training of the generative adversarial network. Results and Discussion: Experiments have demonstrated that the SVGNet proposed in this paper is capable of effectively reducing modal differences, and it increases the matching success rate. Compared to direct image matching, using image generation ideas results in a matching accuracy improvement of more than twice.

Image retrieval for Structure-from-Motion via Graph Convolutional Network

SceneSketcher-v2: Fine-Grained Scene-Level Sketch-Based Image Retrieval Using Adaptive GCNs

Deep Graph Convolutional Quantization Networks for Image Retrieval.

Graph-Based Consistent Matching for Structure-from-Motion

Cross-modal Graph Matching Network for Image-text Retrieval

Multilabel Recognition Algorithm With Multigraph Structure

Complex spatial region representation and similar matching for multi-object image retrieval

Scene Graph Based Fusion Network For Image-Text Retrieval

CNN-G: Convolutional Neural Network Combined With Graph for Image Segmentation With Theoretical Analysis

Text-Image Matching for Cross-Modal Remote Sensing Image Retrieval via Graph Neural Network

Context Spatial Awareness Remote Sensing Image Change Detection Network Based on Graph and Convolution Interaction

SEGCN: Structural Enhancement Graph Clustering Network

Zero-Shot Sketch-Based Image Retrieval via Graph Convolution Network

Semi-supervised constrained graph convolutional network for cross-modal retrieval

Graph Convolution Based Efficient Re-Ranking for Visual Retrieval

Self-SAGCN: Self-Supervised Semantic Alignment for Graph Convolution Network

Graph Structured Network for Image-Text Matching

Structure similarity virtual map generation network for optical and SAR image matching

A graph-based approach for the retrieval of multi-modality medical images

Structure-aware human pose estimation with graph convolutional networks

ImageGCN: Multi-Relational Image Graph Convolutional Networks for Disease Identification with Chest X-rays