Abstract:Registration of pre-operative 3D volumes and intra-operative 2D images is critical for neurological interventions. In various 2D/3D registration tasks, deep learning-based approaches have become popular and achieved tremendous success. However, due to vast space of transformation parameters, estimation errors are significant in these approaches. To tackle above issues, a novel learning-based framework for 2D/3D registration is proposed, consisting of CNN regression and centroid alignment. The former introduces a residual regression network (Res-RegNet) to preliminarily estimate transformation parameters. To further reduce estimation errors, the latter utilizes target vessel centroids to refine projected images. The proposed framework is individually trained and evaluated on three patients, reaching mean Dice of 76.69%, 78.51%, and 85.39%, respectively, all outperforming baseline methods. Extensive ablation studies demonstrate centroid alignment can significantly improve registration performance. As a normalization layer in Res-RegNet, SPADE can modulate activations using binarized inputs through a spatially-adaptive, learned transformation. Semantic information of inputs is preserved to learn better representations for parameter estimation. Moreover, the inference rate of our framework is about 21 FPS combined with the state-of-the-art segmentation model, significantly surpassing real-time requirements (6 $\sim$ 12 FPS) in clinical practice. These promising results indicate the potential of the framework to facilitate various 2D/3D registration tasks. Note to Practitioners—This paper was motivated by the problem of image-guided neurological interventions. Existing 2D/3D registration methods suffer from 1) long iteration times, which are difficult to meet real-time clinical necessities, or 2) significant parameter estimation errors, leading to poor registration accuracies. Therefore, this paper suggests a new registration framework, combining with CNN regression to give predictions of transformation parameters via a single forward propagation, and centroid alignment to reduce estimation errors by translation transformation. The framework is trained and tested on three patients separately and achieves state-of-the-art performance, demonstrating its superiority. Furthermore, the proposed framework is a learning-based method that is adaptable to various image modalities. Therefore, it has latent capacities to be integrated into surgical navigation systems.

Large Scale Unsupervised Brain MRI Image Registration Solution for Learn2Reg 2024

Unsupervised End-to-end Learning for Deformable Medical Image Registration

Unsupervised Multimodal 3D Medical Image Registration with Multilevel Correlation Balanced Optimization

Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning

HCS-Net: Multi-level deformation strategy combined with quadruple attention for image registration

Edge-Aware Pyramidal Deformable Network for Unsupervised Registration of Brain MR Images

Deep learning-based 3D brain multimodal medical image registration

Unsupervised Deformable Image Registration with Absent Correspondences in Pre-operative and Post-Recurrence Brain Tumor MRI Scans

Deformable registration of chest CT images using a 3D convolutional neural network based on unsupervised learning

Robust Image Registration with Absent Correspondences in Pre-operative and Follow-up Brain MRI Scans of Diffuse Glioma Patients

Weakly supervised medical image registration with multi-information guidance

Unsupervised Registration Method based on Deep Neural Network: Application to cardiac and liver MR images

3D MRI Image Super-Resolution for Brain Combining Rigid and Large Diffeomorphic Registration

A New Unsupervised Learning Method for 3D Deformable Medical Image Registration

MsMorph: An Unsupervised pyramid learning network for brain image registration

A Novel Unsupervised Learning Model for Diffeomorphic Image Registration

Brain Tumor Sequence Registration with Non-iterative Coarse-To-Fine Networks and Dual Deep Supervision

Unsupervised Registration for Liver CT-MR Images Based on the Multiscale Integrated Spatial-Weight Module and Dual Similarity Guidance.

Real-Time 2D/3D Registration via CNN Regression and Centroid Alignment

A Coarse-to-Fine Deformable Transformation Framework for Unsupervised Multi-Contrast MR Image Registration with Dual Consistency Constraint

ACSGRegNet: A Deep Learning-based Framework for Unsupervised Joint Affine and Diffeomorphic Registration of Lumbar Spine CT via Cross- and Self-Attention Fusion