Abstract:Multimodal image matching is a core basis for information fusion, change detection, and image-based navigation. However, multimodal images may simultaneously suffer from severe nonlinear radiation distortion (NRD) and complex geometric differences, which pose great challenges to existing methods. Although deep learning-based methods had shown potential in image matching, they mainly focus on same-source images or single types of multimodal images such as optical-synthetic aperture radar (SAR). One of the main obstacles is the lack of public data for different types of multimodal images. In this paper, we make two major contributions to the community of multimodal image matching: First, we collect six typical types of images, including optical-optical, optical-infrared, optical-SAR, optical-depth, optical-map, and nighttime, to construct a multimodal image dataset with a total of 1200 pairs. This dataset has good diversity in image categories, feature classes, resolutions, geometric variations, etc. Second, we propose a scale and rotation invariant feature transform (SRIF) method, which achieves good matching performance without relying on data characteristics. This is one of the advantages of our SRIF over deep learning methods. SRIF obtains the scales of FAST keypoints by projecting them into a simple pyramid scale space, which is based on the study that methods with/without scale space have similar performance under small scale change factors. This strategy largely reduces the complexity compared to traditional Gaussian scale space. SRIF also proposes a local intensity binary transform (LIBT) for SIFT-like feature description, which can largely enhance the structure information inside multimodal images. Extensive experiments on these 1200 image pairs show that our SRIF outperforms current state-of-the-arts by a large margin, including RIFT, CoFSM, LNIFT, and MS-HLMO. Both the created dataset and the code of SRIF will be publicly available in https://github.com/LJY-RS/SRIF.

MLV: Robust Image Matching Via Multi-layer Verification

MLMatch: Multi-layer Image Matching

Leveraging Local Planar Motion Property for Robust Visual Matching and Localization.

A Robust Multi-level Scene Matching Algorithm for Infrared and Visible Light Image

Robust Image Matching Via Local Graph Structure Consensus

A Robust Mismatch Removal Method for Image Matching Based on the Fusion of the Local Features and the Depth

Efficient Covisibility-based Image Matching for Large-Scale SfM

Locally Nonlinear Affine Verification for Multisensor Image Matching

Multiscale Structural Feature Transform for Multi-Modal Image Matching

Robust Feature Matching Via Support-Line Voting and Affine-Invariant Ratios

MIVI: Multi-Stage Feature Matching for Infrared and Visible Image

MV-MOE: A Visual Mixture-of-Experts Model for Optical-SAR Image Matching

Multimodal image matching: A scale-invariant algorithm and an open dataset

An Image Matching Algorithm Based On Singular Value Decomposition and Belief Propagation

Robust Image Matching By Dynamic Feature Selection

Robust Image Matching Algorithm Using Sift on Multiple Layered Strategies

Robust Matching Algorithm For Image Mosaic

Multi-Stage Matching Approach for Mobile Platform Visual Imagery.

Image Matching Via Ranking and Neighborhood Manifold Embedding

Improved HardNet and Stricter Outlier Filtering to Guide Reliable Matching

A Mixture Model for Robust Point Matching under Multi-Layer Motion