Abstract:Registration of pre-operative 3D volumes and intra-operative 2D images is critical for neurological interventions. In various 2D/3D registration tasks, deep learning-based approaches have become popular and achieved tremendous success. However, due to vast space of transformation parameters, estimation errors are significant in these approaches. To tackle above issues, a novel learning-based framework for 2D/3D registration is proposed, consisting of CNN regression and centroid alignment. The former introduces a residual regression network (Res-RegNet) to preliminarily estimate transformation parameters. To further reduce estimation errors, the latter utilizes target vessel centroids to refine projected images. The proposed framework is individually trained and evaluated on three patients, reaching mean Dice of 76.69%, 78.51%, and 85.39%, respectively, all outperforming baseline methods. Extensive ablation studies demonstrate centroid alignment can significantly improve registration performance. As a normalization layer in Res-RegNet, SPADE can modulate activations using binarized inputs through a spatially-adaptive, learned transformation. Semantic information of inputs is preserved to learn better representations for parameter estimation. Moreover, the inference rate of our framework is about 21 FPS combined with the state-of-the-art segmentation model, significantly surpassing real-time requirements (6 $\sim$ 12 FPS) in clinical practice. These promising results indicate the potential of the framework to facilitate various 2D/3D registration tasks. Note to Practitioners—This paper was motivated by the problem of image-guided neurological interventions. Existing 2D/3D registration methods suffer from 1) long iteration times, which are difficult to meet real-time clinical necessities, or 2) significant parameter estimation errors, leading to poor registration accuracies. Therefore, this paper suggests a new registration framework, combining with CNN regression to give predictions of transformation parameters via a single forward propagation, and centroid alignment to reduce estimation errors by translation transformation. The framework is trained and tested on three patients separately and achieves state-of-the-art performance, demonstrating its superiority. Furthermore, the proposed framework is a learning-based method that is adaptable to various image modalities. Therefore, it has latent capacities to be integrated into surgical navigation systems.

Points of interest linear attention network for real‐time non‐rigid liver volume to surface registration

Interactive Liver Segmentation in CT Volumes Using Fully Convolutional Networks.

Non-Rigid Volume to Surface Registration using a Data-Driven Biomechanical Model

Intraoperative laparoscopic liver surface registration with preoperative CT using mixing features and overlapping region masks

Point Cloud Registration in Laparoscopic Liver Surgery Using Keypoint Correspondence Registration Network.

Comparative Analysis of Non-Rigid Registration Techniques for Liver Surface Registration

Learning feature descriptors for pre- and intra-operative point cloud matching for laparoscopic liver registration

Investigating Transformer Encoding Techniques to Improve Data-Driven Volume-to-Surface Liver Registration for Image-Guided Navigation

A Method of Three-Dimensional Non-Rigid Localization of Liver Tumors Based on Structured Light

LIBR+: Improving Intraoperative Liver Registration by Learning the Residual of Biomechanics-Based Deformable Registration

Iterative Reweighted Local Cross Correlation Method for Nonlinear Registration of Multiphase Liver CT Images

Real-time liver tumor localization via a single x-ray projection using deep graph neural network-assisted biomechanical modeling

Residual Aligner-based Network (RAN): Motion-separable structure for coarse-to-fine discontinuous deformable registration

Landmark Tracking in Liver US images Using Cascade Convolutional Neural Networks with Long Short-Term Memory

SPST-CNN: Spatial Pyramid Based Searching and Tagging of Liver’s Intraoperative Live Views Via CNN for Minimal Invasive Surgery

Automatic 3D liver location and segmentation via convolutional neural networks and graph cut

A Robust Automated Markerless Registration Framework for Neurosurgery Navigation.

Real-Time 2D/3D Registration via CNN Regression and Centroid Alignment

Non-rigid image registration with anatomical structure constraint for assessing locoregional therapy of hepatocellular carcinoma

Registration accuracy comparing different rendering techniques on local vs external virtual 3D liver model reconstruction for vascular landmark setting by intraoperative ultrasound in augmented reality navigated liver resection

Structure-aware Registration Network for Liver DCE-CT Images