Abstract:Multi-modal image registration has significant meanings in clinical diagnosis, treatment planning, and image-guided surgery. Since different modalities exhibit different characteristics, finding a fast and accurate correspondence between images of different modalities is still a challenge. In this paper, we propose an image synthesis-based multi-modal registration framework. Image synthesis is performed by a ten-layer fully convolutional network (FCN). The network is composed of 10 convolutional layers combined with batch normalization (BN) and rectified linear unit (ReLU), which can be trained to learn an end-to-end mapping from one modality to the other. After the cross-modality image synthesis, multi-modal registration can be transformed into mono-modal registration. The mono-modal registration can be solved by methods with lower computational complexity, such as sum of squared differences (SSD). We tested our method in T1-weighted vs T2-weighted, T1-weighted vs PD, and T2-weighted vs PD image registrations with BrainWeb phantom data and IXI real patients’ data. The result shows that our framework can achieve higher registration accuracy than the state-of-the-art multi-modal image registration methods, such as local mutual information (LMI) and α-mutual information (α-MI). The average registration errors of our method in experiment with IXI real patients’ data were 1.19, 2.23, and 1.57 compared to 1.53, 2.60, and 2.36 of LMI and 1.34, 2.39, and 1.76 of α-MI in T2-weighted vs PD, T1-weighted vs PD, and T1-weighted vs T2-weighted image registration, respectively. In this paper, we propose an image synthesis-based multi-modal image registration framework. A deep FCN model is developed to perform image synthesis for this framework, which can capture the complex nonlinear relationship between different modalities and discover complex structural representations automatically by a large number of trainable mapping and parameters and perform accurate image synthesis. The framework combined with the deep FCN model and mono-modal registration methods (SSD) can achieve fast and robust results in multi-modal medical image registration. Graphical abstract The workflow of proposed multi-modal image registration framework

Multi-Modal Image Registration Based on Multi-Feature Mutual Information

Regional Weighted Mutual Information For Multimodal Rigid Registration

Local Registration of Medical Images Using Feature Point and Intensity Information

Non-rigid Registration of Multimodality Medical Images Based on ITK

Multi-modal Medical Image Registration Based on Phase Congruency and Quantitative-Qualitative Mutual Information

Multimodality image registration by maximization of mutual information

Registration of Medical Images Using Points and Curves

Deformable multi-modal registration using 3D-FAST conditioned mutual information.

Manifold-based Feature Point Matching for Multi-Modal Image Registration.

Multimodal Medical Image Registration Via Common Representations Learning and Differentiable Geometric Constraints

Multi-source Remote Sensing Image Registration Based on Sift and Optimization of Local Self-Similarity Mutual Information

Influence of Implementation Parameters on Registration of Mr and Spect Brain Images by Maximization of Mutual Information

Mutual information neural estimation for unsupervised multi-modal registration of brain images

Multi-Modality Medical Image Registration Based on Regional Joint Rényi Entropy

A Spatial-Feature-Enhanced MMI Algorithm for Multimodal Airborne Image Registration

Image Synthesis-Based Multi-Modal Image Registration Framework by Using Deep Fully Convolutional Networks

Mutual Information Based Feature Selection for Medical Image Retrieval

Multimodal registration network with multi-scale feature-crossing

Mutual information based 3D registration of rat brain MRI time-series.

Medium-low resolution multisource remote sensing image registration based on SIFT and robust regional mutual information

Feature and Intensity Based Medical Image Registration Using Particle Swarm Optimization