Abstract:Multi-modal image registration has significant meanings in clinical diagnosis, treatment planning, and image-guided surgery. Since different modalities exhibit different characteristics, finding a fast and accurate correspondence between images of different modalities is still a challenge. In this paper, we propose an image synthesis-based multi-modal registration framework. Image synthesis is performed by a ten-layer fully convolutional network (FCN). The network is composed of 10 convolutional layers combined with batch normalization (BN) and rectified linear unit (ReLU), which can be trained to learn an end-to-end mapping from one modality to the other. After the cross-modality image synthesis, multi-modal registration can be transformed into mono-modal registration. The mono-modal registration can be solved by methods with lower computational complexity, such as sum of squared differences (SSD). We tested our method in T1-weighted vs T2-weighted, T1-weighted vs PD, and T2-weighted vs PD image registrations with BrainWeb phantom data and IXI real patients’ data. The result shows that our framework can achieve higher registration accuracy than the state-of-the-art multi-modal image registration methods, such as local mutual information (LMI) and α-mutual information (α-MI). The average registration errors of our method in experiment with IXI real patients’ data were 1.19, 2.23, and 1.57 compared to 1.53, 2.60, and 2.36 of LMI and 1.34, 2.39, and 1.76 of α-MI in T2-weighted vs PD, T1-weighted vs PD, and T1-weighted vs T2-weighted image registration, respectively. In this paper, we propose an image synthesis-based multi-modal image registration framework. A deep FCN model is developed to perform image synthesis for this framework, which can capture the complex nonlinear relationship between different modalities and discover complex structural representations automatically by a large number of trainable mapping and parameters and perform accurate image synthesis. The framework combined with the deep FCN model and mono-modal registration methods (SSD) can achieve fast and robust results in multi-modal medical image registration. Graphical abstract The workflow of proposed multi-modal image registration framework

Deep learning-based affine medical image registration for multimodal minimal-invasive image-guided interventions – A comparative study on generalizability

Multimodal Medical Image Registration Via Common Representations Learning and Differentiable Geometric Constraints

Networks for Joint Affine and Non-parametric Image Registration

A Deep Metric for Multimodal Registration

L2NLF: a novel linear-to-nonlinear framework for multi-modal medical image registration

DISA: DIfferentiable Similarity Approximation for Universal Multimodal Registration

Affine medical image registration with fusion feature mapping in local and global

Towards Accurate and Robust Multi-Modal Medical Image Registration Using Contrastive Metric Learning.

Joint affine and deformable three‐dimensional networks for brain MRI registration

Deformable Medical Image Registration with Global-Local Transformation Network and Region Similarity Constraint.

A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration

Closing the Gap between Deep and Conventional Image Registration using Probabilistic Dense Displacement Networks

Image Synthesis-Based Multi-Modal Image Registration Framework by Using Deep Fully Convolutional Networks

From Local to Global: An Anatomy-Aware Deep Learning Approach for Deformable Registration of Brain MR Images

DELR-Net: a network for 3D multimodal medical image registration in more lightweight application scenarios

An Unsupervised Convolution Neural Network for Deformable Registration of Mono/Multi-Modality Medical Images

Demonstration of accuracy and clinical versatility of mutual information for automatic multimodality image fusion using affine and thin-plate spline warped geometric deformations

Medical Image Registration via Neural Fields

Deep Learning based Inter-Modality Image Registration Supervised by Intra-Modality Similarity

Multimodal Affine Registration for ICGA and MCSL Fundus Images of High Myopia

Affine Medical Image Registration with Coarse-to-Fine Vision Transformer