Abstract:Multimodal image matching is a core basis for information fusion, change detection, and image-based navigation. However, multimodal images may simultaneously suffer from severe nonlinear radiation distortion (NRD) and complex geometric differences, which pose great challenges to existing methods. Although deep learning-based methods had shown potential in image matching, they mainly focus on same-source images or single types of multimodal images such as optical-synthetic aperture radar (SAR). One of the main obstacles is the lack of public data for different types of multimodal images. In this paper, we make two major contributions to the community of multimodal image matching: First, we collect six typical types of images, including optical-optical, optical-infrared, optical-SAR, optical-depth, optical-map, and nighttime, to construct a multimodal image dataset with a total of 1200 pairs. This dataset has good diversity in image categories, feature classes, resolutions, geometric variations, etc. Second, we propose a scale and rotation invariant feature transform (SRIF) method, which achieves good matching performance without relying on data characteristics. This is one of the advantages of our SRIF over deep learning methods. SRIF obtains the scales of FAST keypoints by projecting them into a simple pyramid scale space, which is based on the study that methods with/without scale space have similar performance under small scale change factors. This strategy largely reduces the complexity compared to traditional Gaussian scale space. SRIF also proposes a local intensity binary transform (LIBT) for SIFT-like feature description, which can largely enhance the structure information inside multimodal images. Extensive experiments on these 1200 image pairs show that our SRIF outperforms current state-of-the-arts by a large margin, including RIFT, CoFSM, LNIFT, and MS-HLMO. Both the created dataset and the code of SRIF will be publicly available in https://github.com/LJY-RS/SRIF.

Cross-Domain Matching with Squared-Loss Mutual Information

Cross-Domain Object Matching with Model Selection

Cross-Domain Visual Matching via Generalized Similarity Measure and Feature Learning

Discrete Cross-Modal Hashing for Efficient Multimedia Retrieval

Deep Coupled Metric Learning for Cross-Modal Matching.

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning.

A Multimodal Approach for Cross-Domain Image Retrieval

Cross-Domain Collaborative Learning in Social Multimedia

Cross-validation of matching correlation analysis by resampling matching weights

Cross-modal Deep Metric Learning with Multi-Task Regularization

Cross-Domain Collaborative Learning Via Discriminative Nonparametric Bayesian Model

A unified feature-spatial cycle consistency fusion framework for robust image matching

Matching in the Wild: Learning Anatomical Embeddings for Multi-Modality Images

Cross-Modal Information Maximization for Medical Imaging: CMIM

Unsupervised SAR and Optical Image Matching Using Siamese Domain Adaptation

LSMI-Sinkhorn: Semi-supervised Mutual Information Estimation with Optimal Transport

Cross-Domain Feature Learning in Multimedia

Rate-Distortion Optimized Cross Modal Compression with Multiple Domains

Cross-modal Semantic Interference Suppression for image-text matching

Cross-modal Image Retrieval with Deep Mutual Information Maximization

Multimodal image matching: A scale-invariant algorithm and an open dataset