Abstract:Multispectral and multimodal images are of important usage in the field of multi-source visual information fusion. Due to the alternation or movement of image devices, the acquired multispectral and multimodal images are usually misaligned, and hence image registration is pre-requisite. Different from the registration of common images, the registration of multispectral or multimodal images is a challenging problem due to the nonlinear variation of intensity and gradient. To cope with this challenge, we propose the phase congruency network (PCNet) to enhance the structure similarity of multispectral or multimodal images. The images can then be aligned using the similarity-enhanced feature maps produced by the network. PCNet is constructed under the inspiration of the well-known phase congruency. The network embeds the phase congruency prior into two simple trainable layers and series of modified learnable Gabor kernels. Thanks to the prior knowledge, once trained, PCNet is applicable on a variety of multispectral and multimodal data such as flash/no-flash and RGB/NIR images without additional further tuning. The prior also makes the network lightweight. The trainable parameters of PCNet are 2400 times less than the deep-learning registration method DHN, while its registration performance surpasses DHN. Experimental results validate that PCNet outperforms current state-of-the-art conventional multimodal registration algorithms. Besides, PCNet can act as a complementary part of the deep-learning registration methods, which significantly boosts their registration accuracy. The percentage of the number of images under 1 pixel average corner error (ACE) of UDHN is raised from 0.2% to 89.9% after the processing of PCNet.

Boosting Structure Consistency for Multispectral and Multimodal Image Registration

Block-based Multispectral Image Registration with Application to Spectral Color Measurement

An Accurate Registration Method Based on Global Mixed Structure Similarity (GMSIM) for Remote Sensing Images

Optical Remote Sensing Image Registration Using Spatial-Consistency and Average Regional Information Divergence Minimization Via Quantum-Behaved Particle Swarm Optimization.

Robust Registration of Multimodal Remote Sensing Images with Spectrum Congruency

Multimodal Remote Sensing Image Registration Based on Adaptive Spectrum Congruency

Robust Local Structure Visualization for Remote Sensing Image Registration

Robust Local Feature Descriptor for Multisource Remote Sensing Image Registration

Multi-source Remote Sensing Image Registration Based on Sift and Optimization of Local Self-Similarity Mutual Information

PCNet: A Structure Similarity Enhancement Method for Multispectral and Multimodal Image Registration

Spatial-spectral Cross Correlation for Reliable Multispectral Image Registration.

The Spiking Cortical Model Based Structural Representations for Non-Rigid Multi-Modal Medical Image Registration

Multispectral Image Registration Based on an Improved Scale-Invariant Feature Transform Algorithm

CDFSL: Image Registration for Spaceborne Hyperspectral and Multispectral Data Having Large Spatial-Resolution Difference.

Spatially Coherent Matching for Robust Registration

Robust Multimodal Remote Sensing Image Registration Based on Local Statistical Frequency Information

Multispectral Registration Algorithm Based on Fusion of Infrared and Color Images

Multisensor image registration algorithm combining sift and particle swarm optimization for application in multispectral imagery

SMRD: A Local Feature Descriptor for Multi-modal Image Registration

Multispectral Remote Sensing Image Registration Based on Maximally Stable Extremal Regions

Image registration with structuralized Mutual Information: application to CEST