Visible/Infrared Image Registration Based on Region-Adaptive Contextual Multifeatures.

Qisen Zhao,Liquan Dong,Ming Liu,Lingqin Kong,Xuhong Chu,Mei Hui,Yuejin Zhao

DOI: https://doi.org/10.1109/TGRS.2024.3385088

IF: 8.2

2024-01-01

IEEE Transactions on Geoscience and Remote Sensing

Abstract:Visible and infrared image registration is a challenging problem in computer vision due to the significant differences in appearance and physical properties between the two modalities. A single feature is not enough to remove nonlinear differences, and the matching method faces a trade-off between the high-resolution feature map and the transformer model. In this paper, we propose a novel method called Adaptive-Neighborhood Contextual Multi-features (ANCM-Net) for visible/infrared image registration. Our method addresses the limitations of existing approaches by incorporating depth features and cross-modal similar contour features to form contextual feature representations. Additionally, we propose a region-spanning adaptive cross-attention module to handle low spatial resolution and redundancy in attention computation. This module enables attentional encoding of limited information in the attention location and cross-modal adaptive region through attention region adjustment. In the matching task, we compute an adaptive attention region for each pixel point in the cross-modal image and encode and match the depth features and edge features together. As a result, ANCM-Net not only preserves the long-range dependency of the image feature structure but also achieves fine-grained attention between highly correlated pixels. By extracting cross-modal consistent contextual features to compensate for modality-specific information, our approach improves the cross-modal matching performance. Extensive experiments on real-world captured thermal infrared and visible datasets demonstrate that ANCM-Net outperforms existing image matching methods.