Abstract:Registration of pre-operative 3D volumes and intra-operative 2D images is critical for neurological interventions. In various 2D/3D registration tasks, deep learning-based approaches have become popular and achieved tremendous success. However, due to vast space of transformation parameters, estimation errors are significant in these approaches. To tackle above issues, a novel learning-based framework for 2D/3D registration is proposed, consisting of CNN regression and centroid alignment. The former introduces a residual regression network (Res-RegNet) to preliminarily estimate transformation parameters. To further reduce estimation errors, the latter utilizes target vessel centroids to refine projected images. The proposed framework is individually trained and evaluated on three patients, reaching mean Dice of 76.69%, 78.51%, and 85.39%, respectively, all outperforming baseline methods. Extensive ablation studies demonstrate centroid alignment can significantly improve registration performance. As a normalization layer in Res-RegNet, SPADE can modulate activations using binarized inputs through a spatially-adaptive, learned transformation. Semantic information of inputs is preserved to learn better representations for parameter estimation. Moreover, the inference rate of our framework is about 21 FPS combined with the state-of-the-art segmentation model, significantly surpassing real-time requirements (6 $\sim$ 12 FPS) in clinical practice. These promising results indicate the potential of the framework to facilitate various 2D/3D registration tasks. Note to Practitioners—This paper was motivated by the problem of image-guided neurological interventions. Existing 2D/3D registration methods suffer from 1) long iteration times, which are difficult to meet real-time clinical necessities, or 2) significant parameter estimation errors, leading to poor registration accuracies. Therefore, this paper suggests a new registration framework, combining with CNN regression to give predictions of transformation parameters via a single forward propagation, and centroid alignment to reduce estimation errors by translation transformation. The framework is trained and tested on three patients separately and achieves state-of-the-art performance, demonstrating its superiority. Furthermore, the proposed framework is a learning-based method that is adaptable to various image modalities. Therefore, it has latent capacities to be integrated into surgical navigation systems.

3Dcnn-Based Palpation Localization with Temporal Attention Module.

Two-dimensional discrete feature based spatial attention CapsNet For sEMG signal recognition

Cardiac Arrhythmia Classification Based on 3D Recurrence Plot Analysis and Deep Learning.

Palpation Localization of Radial Artery Based on 3-Dimensional Convolutional Neural Networks

Automated Heartbeat Classification Using 3-D Inputs Based on Convolutional Neural Network with Multi-Fields of View.

Accurate Targeting in Robot-Assisted TCM Pulse Diagnosis Using Adaptive Sensor Fusion

Automated Heartbeat Classification Exploiting Convolutional Neural Network with Channel-Wise Attention

3D Localization of Hand Acupoints Using Hand Geometry and Landmark Points Based on RGB-D CNN Fusion

Intelligent Detection of Pulse Position in Traditional Chinese Medicine Based on Bionic Pulse Diagnose Robotics

Physiology and Geometry-Informed Deep Learning Framework for 12-Lead ECG Cardiac Transmembrane Potential Recovery

Full-field Wrist Pulse Signal Acquisition and Analysis by 3D Digital Image Correlation

Automated ECG classification using a non-local convolutional block attention module

A Protocol for Digitalized Collection of Traditional Chinese Medicine (TCM) Pulse Information Using Bionic Pulse Diagnosis Equipment

Real-time monitoring of manual acupuncture stimulation parameters based on domain adaptive 3D hand pose estimation

Pioneering Data Processing for Convolutional Neural Networks to Enhance the Diagnostic Accuracy of Traditional Chinese Medicine Pulse Diagnosis for Diabetes

An Image-Based Facial Acupoint Detection Approach Using High-Resolution Network and Attention Fusion

Pulse localization networks with infrared camera

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation

Real-time location of acupuncture points based on anatomical landmarks and pose estimation models

A Paralleled CNN and Transformer Network for PPG-based Cuff-Less Blood Pressure Estimation

Real-Time 2D/3D Registration via CNN Regression and Centroid Alignment