Abstract:Registration of pre-operative 3D volumes and intra-operative 2D images is critical for neurological interventions. In various 2D/3D registration tasks, deep learning-based approaches have become popular and achieved tremendous success. However, due to vast space of transformation parameters, estimation errors are significant in these approaches. To tackle above issues, a novel learning-based framework for 2D/3D registration is proposed, consisting of CNN regression and centroid alignment. The former introduces a residual regression network (Res-RegNet) to preliminarily estimate transformation parameters. To further reduce estimation errors, the latter utilizes target vessel centroids to refine projected images. The proposed framework is individually trained and evaluated on three patients, reaching mean Dice of 76.69%, 78.51%, and 85.39%, respectively, all outperforming baseline methods. Extensive ablation studies demonstrate centroid alignment can significantly improve registration performance. As a normalization layer in Res-RegNet, SPADE can modulate activations using binarized inputs through a spatially-adaptive, learned transformation. Semantic information of inputs is preserved to learn better representations for parameter estimation. Moreover, the inference rate of our framework is about 21 FPS combined with the state-of-the-art segmentation model, significantly surpassing real-time requirements (6 $\sim$ 12 FPS) in clinical practice. These promising results indicate the potential of the framework to facilitate various 2D/3D registration tasks. Note to Practitioners—This paper was motivated by the problem of image-guided neurological interventions. Existing 2D/3D registration methods suffer from 1) long iteration times, which are difficult to meet real-time clinical necessities, or 2) significant parameter estimation errors, leading to poor registration accuracies. Therefore, this paper suggests a new registration framework, combining with CNN regression to give predictions of transformation parameters via a single forward propagation, and centroid alignment to reduce estimation errors by translation transformation. The framework is trained and tested on three patients separately and achieves state-of-the-art performance, demonstrating its superiority. Furthermore, the proposed framework is a learning-based method that is adaptable to various image modalities. Therefore, it has latent capacities to be integrated into surgical navigation systems.

Multi-View Point-Based Registration for Native Knee Kinematics Measurement with Feature Transfer Learning

Transfer learning from an artificial radiograph-landmark dataset for registration of the anatomic skull model to dual fluoroscopic X-ray images

Fusion of Multimodality Image and Point Cloud for Spatial Surface Registration for Knee Arthroplasty

Networks for Joint Affine and Non-parametric Image Registration

Scan-Free and Fully Automatic Tracking of Native Knee Anatomy from Dynamic Stereo-Radiography with Statistical Shape and Intensity Models

Machine learning for automated and real-time two-dimensional to three-dimensional registration of the spine using a single radiograph

A Robust and Interpretable Deep Learning Framework for Multi-modal Registration via Keypoints

A marker-less human motion analysis system for motion-based biomarker identification and quantification in knee disorders

Real-Time 2D/3D Registration via CNN Regression and Centroid Alignment

Accurate Three-Dimensional Registration of Magnetic Resonance Images for Detecting Local Changes in Cartilage Thickness.

Integration of statistical shape modeling and alternating interpolation-based model tracking technique for measuring knee kinematics in vivo using clinical interleaved bi-plane fluoroscopy

Multi-step 3D/2D Image Registration for Image-guided Spinal Surgery

Transformer-based 2D/3D medical image registration for X-ray to CT via anatomical features

Robust and efficient 3D registration via depth map-based feature point matching in image-guided neurosurgery

A Two-Step Deep Learning Method for 3DCT-2DUS Kidney Registration During Breathing

A marker-less human motion analysis system for motion-based biomarker discovery in knee disorders

Bio-Measurements Estimation and Support in Knee Recovery through Machine Learning

2D/3D registration based on biplanar X-ray and CT images for surgical navigation

Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation

Dual output feature fusion networks for femoral segmentation and quantitative analysis of the knee joint

Volumetric feature points integration with bio-structure-informed guidance for deformable multi-modal CT image registration