Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing Images
Dou Quan,Huiyuan Wei,Shuang Wang,Yi Li,Jocelyn Chanussot,Yanhe Guo,Biao Hou,Licheng Jiao
DOI: https://doi.org/10.1109/jstars.2023.3276409
IF: 4.715
2023-05-30
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Abstract:Deep convolutional networks are powerful for local feature learning and have shown advantages in image matching and registration. However, the significant differences between cross-modal images increase the challenge of image registration. The deep network should extract modality-invariant features to identify the matching samples and discriminative features to separate the nonmatching samples. The deep network can extract features invariant to the image modality changes by multiple nonlinear mapping layers. However, it does not inevitably lose rich details and affect the discrimination of features, degrading registration performances. This article proposes a novel deep wavelet learning network (DW-Net) for local feature learning. It incorporates spectral information into deep convolutional features for improving cross-modal image matching and registration. Specifically, this article aims to learn the multiresolution wavelet features through multilevel wavelet transform (WT) and the convolutional network. The cross-modal images are divided into low-frequency and high-frequency parts through WT. DW-Net can adaptively extract the shared features from the low-frequency part and useful details from the high-frequency part, which can enhance the modality invariance and discrimination of features. Additionally, the multiresolution wavelet features contain multiscale information and contribute to improving the matching accuracy. Extensive experiments demonstrate the significant advantages in terms of the accuracy and robustness of DW-Net on cross-modal remote sensing image registration. DW-Net can increase the image patch matching accuracy by 3.7% and improve image registration probability by 12.1%. Moreover, DW-Net shows strong generalization performances from low resolution to high resolution and from optical– synthetic aperture radar to other cross-modal image registration.
imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geography, physical