Abstract:<p>Multimodal image matching, which refers to identifying and then corresponding the same or similar structure/content from two or more images that are of significant modalities or nonlinear appearance difference, is a fundamental and critical problem in a wide range of applications, including medical, remote sensing and computer vision. An increasing number and diversity of methods have been proposed over the past decades, particularly in this deep learning era, due to the challenges in eliminating modality variance and geometrical deformation that intrinsically exist in multimodal image matching. However, a comprehensive review and analysis of traditional and recent trainable methods and their applications in different research fields are lacking. To this end and in this survey, we first introduce two general frameworks, saying area- and feature-based, in terms of their core components, taxonomy, and procedure details. Second, we provide a comprehensive review of multimodal image matching methods from handcrafted to deep methods for each research field according to their imaging nature, including medical, remote sensing and computer vision. Extensive experimental comparisons of interest point detection, description and matching, and image registration are performed on various datasets containing common types of multimodal image pairs that we collected and annotated. Finally, we briefly introduce and analyze several typical applications to reveal the significance of multimodal image matching and provide insightful discussions and conclusions to these multimodal image matching approaches, and simultaneously deliver their future trends for researchers and engineers in related research areas to achieve further breakthroughs.</p>

Multimodal Image Alignment via Linear Mapping between Feature Modalities

Multimodal Image Alignment Through a Multiscale Chain of Neural Networks with Application to Remote Sensing

A Multimodal Image Matching Algorithm Based on Structure Feature

Multimodal Medical Image Registration Via Common Representations Learning and Differentiable Geometric Constraints

Image Registration Method for Multimodal Images

Fast and Robust Matching for Multimodal Remote Sensing Image Registration

Multispectral Image Alignment with Nonlinear Scale-Invariant Keypoint and Enhanced Local Feature Matrix

Laplacian feature detection and feature alignment for multimodal ophthalmic image registration using phase correlation and Hessian affine feature space

Automated Registration of Multimodality Images by Maximization of A Region Similarity Measure

Multi-modal Image Matching Based on Local Frequency Information

Multimodal Remote Sensing Image Registration Based on Image Transfer and Local Features

Multimodal Remote Sensing Image Matching via Learning Features and Attention Mechanism

Landmark Image Retrieval by Jointing Feature Refinement and Multimodal Classifier Learning.

Content-Based Visual Landmark Search Via Multimodal Hypergraph Learning.

Multimodal image registration based on orientation-moment

Alternative Telescopic Displacement: An Efficient Multimodal Alignment Method

Variational Methods for Multimodal Image Matching

Multimodal Features Alignment for Vision–Language Object Tracking

A review of multimodal image matching: Methods and applications

Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity