Abstract:Accurate multispectral image matching presents significant challenges due to non-linear intensity variations across spectral modalities, extreme viewpoint changes, and the scarcity of labeled datasets. Current state-of-the-art methods are typically specialized for a single spectral difference, such as visibleinfrared, and struggle to adapt to other modalities due to their reliance on expensive supervision, such as depth maps or camera poses. To address the need for rapid adaptation across modalities, we introduce XPoint, a self-supervised, modular image-matching framework designed for adaptive training and fine-tuning on aligned multispectral datasets, allowing users to customize key components based on their specific tasks. XPoint employs modularity and self-supervision to allow for the adjustment of elements such as the base detector, which generates pseudoground truth keypoints invariant to viewpoint and spectrum variations. The framework integrates a VMamba encoder, pretrained on segmentation tasks, for robust feature extraction, and includes three joint decoder heads: two are dedicated to interest point and descriptor extraction; and a task-specific homography regression head imposes geometric constraints for superior performance in tasks like image registration. This flexible architecture enables quick adaptation to a wide range of modalities, demonstrated by training on Optical-Thermal data and fine-tuning on settings such as visual-near infrared, visual-infrared, visual-longwave infrared, and visual-synthetic aperture radar. Experimental results show that XPoint consistently outperforms or matches state-ofthe-art methods in feature matching and image registration tasks across five distinct multispectral datasets. Our source code is available at <a class="link-external link-https" href="https://github.com/canyagmur/XPoint" rel="external noopener nofollow">this https URL</a>.

Heterogeneous self-supervised interest point matching for multi-modal remote sensing image registration

Self-Supervised Keypoint Detection and Cross-Fusion Matching Networks for Multimodal Remote Sensing Image Registration

Multi-source Remote Sensing Image Registration Based on Sift and Optimization of Local Self-Similarity Mutual Information

A Novel Local Pettern Based Self-Similarity Descriptor for Multisource Remote Sensing Image Registration.

A Semi-Supervised Image Registration Framework Based on Multimodal Cross-Attention

XPoint: A Self-Supervised Visual-State-Space based Architecture for Multispectral Image Registration

Remote Sensing Image Registration Based on Multi-Head Self-Attention Mechanism

Robust Local Feature Descriptor for Multisource Remote Sensing Image Registration

A Fast and Fully Automatic Registration Approach Based on Point Features for Multi-Source Remote-Sensing Images

A Fast and Robust Matching System for Multimodal Remote Sensing Image Registration

Multifeature Alignment and Matching Network for SAR and Optical Image Registration

Robust registration of multi-modal remote sensing images based on multi-dimensional oriented self-similarity features

Fast and Robust Matching for Multimodal Remote Sensing Image Registration

Gradient Weakly Sensitive Multi-Source Sensor Image Registration Method

Mutual Information Based Multi-Modal Remote Sensing Image Registration Using Adaptive Feature Weight

Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity

A New Fast Multi-Source Remote Sensing Image Registration Algorithm

Deep Generative Matching Network for Optical and SAR Image Registration

Multimodal Remote Sensing Image Registration Based on Adaptive Spectrum Congruency

Multimodal Remote Sensing Image Matching via Learning Features and Attention Mechanism

Multi-Modal Image Registration Based on Local Self-Similarity and Bidirectional Matching