Abstract:Registration of pre-operative 3D volumes and intra-operative 2D images is critical for neurological interventions. In various 2D/3D registration tasks, deep learning-based approaches have become popular and achieved tremendous success. However, due to vast space of transformation parameters, estimation errors are significant in these approaches. To tackle above issues, a novel learning-based framework for 2D/3D registration is proposed, consisting of CNN regression and centroid alignment. The former introduces a residual regression network (Res-RegNet) to preliminarily estimate transformation parameters. To further reduce estimation errors, the latter utilizes target vessel centroids to refine projected images. The proposed framework is individually trained and evaluated on three patients, reaching mean Dice of 76.69%, 78.51%, and 85.39%, respectively, all outperforming baseline methods. Extensive ablation studies demonstrate centroid alignment can significantly improve registration performance. As a normalization layer in Res-RegNet, SPADE can modulate activations using binarized inputs through a spatially-adaptive, learned transformation. Semantic information of inputs is preserved to learn better representations for parameter estimation. Moreover, the inference rate of our framework is about 21 FPS combined with the state-of-the-art segmentation model, significantly surpassing real-time requirements (6 $\sim$ 12 FPS) in clinical practice. These promising results indicate the potential of the framework to facilitate various 2D/3D registration tasks. Note to Practitioners—This paper was motivated by the problem of image-guided neurological interventions. Existing 2D/3D registration methods suffer from 1) long iteration times, which are difficult to meet real-time clinical necessities, or 2) significant parameter estimation errors, leading to poor registration accuracies. Therefore, this paper suggests a new registration framework, combining with CNN regression to give predictions of transformation parameters via a single forward propagation, and centroid alignment to reduce estimation errors by translation transformation. The framework is trained and tested on three patients separately and achieves state-of-the-art performance, demonstrating its superiority. Furthermore, the proposed framework is a learning-based method that is adaptable to various image modalities. Therefore, it has latent capacities to be integrated into surgical navigation systems.

A preliminary exploration into top-down and bottom-up deep-learning approaches to localising neuro-interventional point targets in volumetric MRI

A model-based, semi-global segmentation approach for automatic 3-D point landmark localization in neuroimages

Enhancing Precision in Medical Imaging: A 3D CNN Approach for Fiducial Point Detection in MRI Data

From Local to Global: An Anatomy-Aware Deep Learning Approach for Deformable Registration of Brain MR Images

Deep learning-based automatic pipeline for 3D needle localization on intra-procedural 3D MRI

Real-Time 2D/3D Registration via CNN Regression and Centroid Alignment

Automatic Localization of Key Structures for Subthalamic Nucleus-Deep Brain Stimulation Surgery via Prior-Enhanced Multi-Object Magnetic Resonance Imaging Segmentation

An Image Quality Transfer Technique for Localising Deep Brain Stimulation Targets

Automatic localization of target point for STN‐DBS via hierarchical Attention‐UNet based MRI segmentation

Non-invasive Localization of the Ventricular Excitation Origin Without Patient-specific Geometries Using Deep Learning

Deep learning trends for focal brain pathology segmentation in MRI

Multi-dimensional topological loss for cortical plate segmentation in fetal brain MRI

Uncertainty Estimation in Medical Image Localization: Towards Robust Anterior Thalamus Targeting for Deep Brain Stimulation

A Novel Deep Recurrent Convolutional Neural Network for Subthalamic Nucleus Localization Using Local Field Potential Signals

Localization of the Epileptic Focus using Multi-scale Deep Learning

A Systematic Approach for MRI Brain Tumor Localization and Segmentation Using Deep Learning and Active Contouring

Local and Deep Features Based Convolutional Neural Network Frameworks for Brain MRI Anomaly Detection

Hough-CNN: Deep learning for segmentation of deep brain regions in MRI and ultrasound

Toward defining deep brain stimulation targets in MNI space: A subcortical atlas based on multimodal MRI, histology and structural connectivity

Spherical-Patches Extraction for Deep-Learning-Based Critical Points Detection in 3D Neuron Microscopy Images

Localising the Seizure Onset Zone from Single-Pulse Electrical Stimulation Responses with a CNN Transformer