Abstract:Multi-modal image registration has significant meanings in clinical diagnosis, treatment planning, and image-guided surgery. Since different modalities exhibit different characteristics, finding a fast and accurate correspondence between images of different modalities is still a challenge. In this paper, we propose an image synthesis-based multi-modal registration framework. Image synthesis is performed by a ten-layer fully convolutional network (FCN). The network is composed of 10 convolutional layers combined with batch normalization (BN) and rectified linear unit (ReLU), which can be trained to learn an end-to-end mapping from one modality to the other. After the cross-modality image synthesis, multi-modal registration can be transformed into mono-modal registration. The mono-modal registration can be solved by methods with lower computational complexity, such as sum of squared differences (SSD). We tested our method in T1-weighted vs T2-weighted, T1-weighted vs PD, and T2-weighted vs PD image registrations with BrainWeb phantom data and IXI real patients’ data. The result shows that our framework can achieve higher registration accuracy than the state-of-the-art multi-modal image registration methods, such as local mutual information (LMI) and α-mutual information (α-MI). The average registration errors of our method in experiment with IXI real patients’ data were 1.19, 2.23, and 1.57 compared to 1.53, 2.60, and 2.36 of LMI and 1.34, 2.39, and 1.76 of α-MI in T2-weighted vs PD, T1-weighted vs PD, and T1-weighted vs T2-weighted image registration, respectively. In this paper, we propose an image synthesis-based multi-modal image registration framework. A deep FCN model is developed to perform image synthesis for this framework, which can capture the complex nonlinear relationship between different modalities and discover complex structural representations automatically by a large number of trainable mapping and parameters and perform accurate image synthesis. The framework combined with the deep FCN model and mono-modal registration methods (SSD) can achieve fast and robust results in multi-modal medical image registration. Graphical abstract The workflow of proposed multi-modal image registration framework

A synthetic registration method for multi-modal images based on dynamic credibility

Regional Weighted Mutual Information For Multimodal Rigid Registration

A Fully Automatic Registration Approach Based on Contour and SIFT for HJ-1 Images

Multi-source Remote Sensing Image Registration Based on Sift and Optimization of Local Self-Similarity Mutual Information

Infrared and Visible Image Registration in UAV Inspection

Multi-sensor Images Registration Based on SIFT and Extended Phase Correlation

A Novel Coarse-to-Fine Scheme for Remote Sensing Image Registration Based on SIFT and Phase Correlation

An Iterative SIFT Based on Intensity and Spatial Information for Remote Sensing Image Registration

A Robust Infrared and Visible Image Registration Method for Dual Sensor UAV System

Multi-Modal Image Registration Based on Multi-Feature Mutual Information

Multimodal Medical Image Registration Via Common Representations Learning and Differentiable Geometric Constraints

Assembly Method for Multimodal Image Registration

Automatic Remote Sensing Image Registration Algorithm Based on SIFT

Influence of Implementation Parameters on Registration of Mr and Spect Brain Images by Maximization of Mutual Information

Automatic Remote Sensing Image Registration Based on VTS-PCMIC Algorithm

Mutual Information-Based Registration of Temporal and Stereo Retinal Images Using Constrained Optimization.

An Accurate Registration Method Based on Global Mixed Structure Similarity (GMSIM) for Remote Sensing Images

Medium-low resolution multisource remote sensing image registration based on SIFT and robust regional mutual information

Robust and Efficient Registration of Infrared and Visible Images for Vehicular Imaging Systems

Image Synthesis-Based Multi-Modal Image Registration Framework by Using Deep Fully Convolutional Networks

Multimodal Remote Sensing Image Registration Based on Adaptive Multi-scale PIIFD