Abstract:Optical and synthetic aperture radar (SAR) image registration is important for multimodal remote sensing image information fusion. Recently, deep matching networks have shown better performances than traditional methods of image matching. However, due to significant differences between optical and SAR images, the performances of existing deep learning methods still need to be further improved. This article proposes a self-distillation feature learning network (SDNet) for optical and SAR image registration, improving performance from network structure and network optimization. First, we explore the impact of different weight-sharing strategies on optical and SAR image matching. Then, we design a partially unshared feature learning network for multimodal image feature learning. It has fewer parameters than the fully unshared network and has more flexibility than the fully shared network. In addition, the limited binary supervised information (matching or nonmatching) is insufficient to train the deep matching networks for optical-SAR image registration. Thus, we propose a self-distillation feature learning method to exploit more similarity information for deep network optimization enhancement, such as the similarity ordering between a series of nonmatching patch pairs. The exploited rich similarity information will significantly enhance network training and improve matching accuracy. Finally, existing deep learning methods brute-force make the matching features of the optical and SAR image patches similar, which will lead to the loss of discriminative information and degeneration of the matching performances. Thus, we build an auxiliary task reconstruction learning to optimize the feature learning network to keep more discriminative information. Extensive experiments demonstrate the effectiveness of our proposed method on multimodal image registration.

Deep modality independent descriptor learning for optical and sar image patch matching

Deep Continuous Matching Network for More Robust Multi-Modal Remote Sensing Image Patch Matching

Self-Distillation Feature Learning Network for Optical and SAR Image Registration

Deep Feature Correlation Learning for Multi-Modal Remote Sensing Image Registration

SAR-optical Feature Matching: A Large-Scale Patch Dataset and a Deep Local Descriptor

A Feature Decomposition Framework for Multi-Modal Image Patch Matching

Shape-Adaptive Modality Independent Region Descriptor for Multimodal Remote Sensing Image Matching

A Novel Local Pettern Based Self-Similarity Descriptor for Multisource Remote Sensing Image Registration.

Deep Generative Matching Network for Optical and SAR Image Registration

Robust Local Feature Descriptor for Multisource Remote Sensing Image Registration

Learning Local Event-based Descriptor for Patch-based Stereo Matching

Multimodal Remote Sensing Image Matching via Learning Features and Attention Mechanism

Feature Matching and Position Matching Between Optical and SAR with Local Deep Feature Descriptor

SSML-QNet: Scale-Separative Metric Learning Quadruplet Network for Multi-Modal Image Patch Matching

Optical and SAR Image Matching Using Pixelwise Deep Dense Features

Explore Better Network Framework for High-Resolution Optical and SAR Image Matching

Fourier Domain Adaptive Multi-Modal Remote Sensing Image Template Matching Based on Siamese Network

Metric Learning for 2D Image Patch and 3D Point Cloud Volume Matching.

Robust Multi-Sensor Image Matching Based on Normalized Self-Similarity Region Descriptor

Deep Coupled Metric Learning for Cross-Modal Matching.

Registration of Multimodal Remote Sensing Image Based on Deep Fully Convolutional Neural Network