Abstract:Multimodal remote sensing images with large rotation transformation (RT) are challenging to be registered. It needs to deal with the global geometric deformation caused by great RT and significant local appearance differences caused by different imaging mechanisms. Existing deep learning methods mainly use a single deep descriptor learning (DDL) network to extract invariant features for identifying matching samples and discriminative feature descriptors for separating nonmatching samples. However, it is difficult to extract local invariant feature descriptors to RT and modality change through a single DDL network. This article proposes a novel coarse-to-fine deep learning image registration framework for multimodal remote sensing images based on two task-specific deep models. Specifically, in the coarse registration stage, this article designs an effective deep ordinal regression (DOR) network for rotation correction, which can reduce the difficulty of multimodal image registration and boost image registration. The proposed DOR network transforms the rotation correction task into a rotation ordinal regression problem, which can exploit the potential relationship between the rotation ordinals to improve the accuracy of rotation estimation. In the fine registration stage, we adopt the DDL network to deal with the image modality change based on the rotation-corrected images. Extensive experimental results on multimodal image datasets demonstrate the significant advantages of the proposed coarse-to-fine deep learning registration framework. The DOR network achieves higher rotation correction accuracy, which can significantly improve the multimodal image registration performances.

Robust Multimodal Image Registration Using Deep Recurrent Reinforcement Learning

End-to-end multimodal image registration via reinforcement learning

A Spatiotemporal Agent for Robust Multimodal Registration

Multimodal Medical Image Registration Via Common Representations Learning and Differentiable Geometric Constraints

Synergistic registration of CT-MRI brain images and retinal images: A novel approach leveraging reinforcement learning and modified artificial rabbit optimization

Contextual Reinforcement Learning for Unsupervised Deformable Multimodal Medical Images Registration

Attention Guided Policy Optimization for 3D Medical Image Registration

Deep learning-based 3D brain multimodal medical image registration

An Artificial Agent for Robust Image Registration

Towards Accurate and Robust Multi-Modal Medical Image Registration Using Contrastive Metric Learning.

Deep Learning based Inter-Modality Image Registration Supervised by Intra-Modality Similarity

Registration of Multimodal Remote Sensing Image Based on Deep Fully Convolutional Neural Network

Networks for Joint Affine and Non-parametric Image Registration

MvMM-RegNet: A new image registration framework based on multivariate mixture model and neural network estimation

Unsupervised Multimodal 3D Medical Image Registration with Multilevel Correlation Balanced Optimization

Image registration using machine and deep learning

Mutualreg: Mutual Learning for Unsupervised Medical Image Registration

Reinforcement Learning Driven Intra-modal and Inter-modal Representation Learning for 3D Medical Image Classification

CMR-Agent: Learning a Cross-Modal Agent for Iterative Image-to-Point Cloud Registration

DELR-Net: a network for 3D multimodal medical image registration in more lightweight application scenarios

A Novel Coarse-to-Fine Deep Learning Registration Framework for Multimodal Remote Sensing Images.