Abstract:Registration of pre-operative 3D volumes and intra-operative 2D images is critical for neurological interventions. In various 2D/3D registration tasks, deep learning-based approaches have become popular and achieved tremendous success. However, due to vast space of transformation parameters, estimation errors are significant in these approaches. To tackle above issues, a novel learning-based framework for 2D/3D registration is proposed, consisting of CNN regression and centroid alignment. The former introduces a residual regression network (Res-RegNet) to preliminarily estimate transformation parameters. To further reduce estimation errors, the latter utilizes target vessel centroids to refine projected images. The proposed framework is individually trained and evaluated on three patients, reaching mean Dice of 76.69%, 78.51%, and 85.39%, respectively, all outperforming baseline methods. Extensive ablation studies demonstrate centroid alignment can significantly improve registration performance. As a normalization layer in Res-RegNet, SPADE can modulate activations using binarized inputs through a spatially-adaptive, learned transformation. Semantic information of inputs is preserved to learn better representations for parameter estimation. Moreover, the inference rate of our framework is about 21 FPS combined with the state-of-the-art segmentation model, significantly surpassing real-time requirements (6 $\sim$ 12 FPS) in clinical practice. These promising results indicate the potential of the framework to facilitate various 2D/3D registration tasks. Note to Practitioners—This paper was motivated by the problem of image-guided neurological interventions. Existing 2D/3D registration methods suffer from 1) long iteration times, which are difficult to meet real-time clinical necessities, or 2) significant parameter estimation errors, leading to poor registration accuracies. Therefore, this paper suggests a new registration framework, combining with CNN regression to give predictions of transformation parameters via a single forward propagation, and centroid alignment to reduce estimation errors by translation transformation. The framework is trained and tested on three patients separately and achieves state-of-the-art performance, demonstrating its superiority. Furthermore, the proposed framework is a learning-based method that is adaptable to various image modalities. Therefore, it has latent capacities to be integrated into surgical navigation systems.

Initialize globally before acting locally: Enabling Landmark-free 3D US to MRI Registration

An Image Registration Approach Based on 3D Geometric Projection Similarity of the Human Head

Landmark-based elastic registration using approximating thin-plate splines

A model-based, semi-global segmentation approach for automatic 3-D point landmark localization in neuroimages

Landmark Optimization Using Local Curvature for Point-Based Nonlinear Rodent Brain Image Registration

Robust Multi-scale Anatomical Landmark Detection in Incomplete 3D-CT Data

A Robust Automated Markerless Registration Framework for Neurosurgery Navigation.

A Registration and Deep Learning Approach to Automated Landmark Detection for Geometric Morphometrics

Automatic Landmarks Correspondence Detection in Medical Images with an Application to Deformable Image Registration

Robust and efficient 3D registration via depth map-based feature point matching in image-guided neurosurgery

Rigid Registration of Freehand 3D Ultrasound and CT-Scan Kidney Images

Non-rigid registration of 3D ultrasound for neurosurgery using automatic feature detection and matching

Automatic landmark annotation and dense correspondence registration for 3D human facial images

Three-Dimensional Multi-Modality Registration for Orthopaedics and Cardiovascular Settings: State-of-the-Art and Clinical Applications

Real-Time 2D/3D Registration via CNN Regression and Centroid Alignment

A New Robust Markerless Method for Automatic Image-to-patient Registration in Image-Guided Neurosurgery System.

Landmark And Intensity-Based Images Elastic Registration Using Hierarchical B-Splines

Automatic 3D Intersubject Registration of MR Volumetric Data in Standardized Talairach Space

Registration of IRT and visible light images in neurosurgery: analysis and comparison of automatic intensity-based registration approaches

LandmarkNet: a 2D Digital Radiograph Landmark Estimator for Registration