Abstract:Optical and synthetic aperture radar (SAR) image registration is important for multimodal remote sensing image information fusion. Recently, deep matching networks have shown better performances than traditional methods of image matching. However, due to significant differences between optical and SAR images, the performances of existing deep learning methods still need to be further improved. This article proposes a self-distillation feature learning network (SDNet) for optical and SAR image registration, improving performance from network structure and network optimization. First, we explore the impact of different weight-sharing strategies on optical and SAR image matching. Then, we design a partially unshared feature learning network for multimodal image feature learning. It has fewer parameters than the fully unshared network and has more flexibility than the fully shared network. In addition, the limited binary supervised information (matching or nonmatching) is insufficient to train the deep matching networks for optical-SAR image registration. Thus, we propose a self-distillation feature learning method to exploit more similarity information for deep network optimization enhancement, such as the similarity ordering between a series of nonmatching patch pairs. The exploited rich similarity information will significantly enhance network training and improve matching accuracy. Finally, existing deep learning methods brute-force make the matching features of the optical and SAR image patches similar, which will lead to the loss of discriminative information and degeneration of the matching performances. Thus, we build an auxiliary task reconstruction learning to optimize the feature learning network to keep more discriminative information. Extensive experiments demonstrate the effectiveness of our proposed method on multimodal image registration.

An Attention-Based Fusion for Handcrafted and Deep Feature to Improve Optical and SAR Image Matching

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

Fusion Algorithm of Laser-Point Clouds and Optical Images

Robust Optical and SAR Image Matching Using Attention-Enhanced Structural Features

Image Matching and Localization Based on Fusion of Handcrafted and Deep Features

Explore Better Network Framework for High-Resolution Optical and SAR Image Matching

Feature Fusion Classification for Optical Image and SAR Image Based on Spatial-spectral Attention

SAR-optical Feature Matching: A Large-Scale Patch Dataset and a Deep Local Descriptor

Feature Matching and Position Matching Between Optical and SAR with Local Deep Feature Descriptor

Optical and SAR Image Matching Using Pixelwise Deep Dense Features

SAR and Optical Image Registration Based on Deep Learning with Co-Attention Matching Module

A lightweight deep convolutional network with inverted residuals for matching optical and SAR images

Self-Distillation Feature Learning Network for Optical and SAR Image Registration

Feature-level image fusion for SAR and optical images

Learning SAR-Optical Cross Modal Features for Land Cover Classification

DAFCNN: A Dual-Channel Feature Extraction and Attention Feature Fusion Convolution Neural Network for SAR Image and MS Image Fusion

A Deep Learning-Based Heterogeneous Spatio-Temporal-Spectral Fusion: SAR and Optical Images.

Infrared and Visible Image Fusion Method Based on Hierarchical Attention Mechanism

Shared contents alignment across multiple granularities for robust SAR-optical image matching

SAR-Optical Image Matching by Integrating Siamese U-Net with FFT Correlation

Optical and SAR Image Fusion Based on Complementary Feature Decomposition and Visual Saliency Features