Abstract:Registration of pre-operative 3D volumes and intra-operative 2D images is critical for neurological interventions. In various 2D/3D registration tasks, deep learning-based approaches have become popular and achieved tremendous success. However, due to vast space of transformation parameters, estimation errors are significant in these approaches. To tackle above issues, a novel learning-based framework for 2D/3D registration is proposed, consisting of CNN regression and centroid alignment. The former introduces a residual regression network (Res-RegNet) to preliminarily estimate transformation parameters. To further reduce estimation errors, the latter utilizes target vessel centroids to refine projected images. The proposed framework is individually trained and evaluated on three patients, reaching mean Dice of 76.69%, 78.51%, and 85.39%, respectively, all outperforming baseline methods. Extensive ablation studies demonstrate centroid alignment can significantly improve registration performance. As a normalization layer in Res-RegNet, SPADE can modulate activations using binarized inputs through a spatially-adaptive, learned transformation. Semantic information of inputs is preserved to learn better representations for parameter estimation. Moreover, the inference rate of our framework is about 21 FPS combined with the state-of-the-art segmentation model, significantly surpassing real-time requirements (6 $\sim$ 12 FPS) in clinical practice. These promising results indicate the potential of the framework to facilitate various 2D/3D registration tasks. Note to Practitioners—This paper was motivated by the problem of image-guided neurological interventions. Existing 2D/3D registration methods suffer from 1) long iteration times, which are difficult to meet real-time clinical necessities, or 2) significant parameter estimation errors, leading to poor registration accuracies. Therefore, this paper suggests a new registration framework, combining with CNN regression to give predictions of transformation parameters via a single forward propagation, and centroid alignment to reduce estimation errors by translation transformation. The framework is trained and tested on three patients separately and achieves state-of-the-art performance, demonstrating its superiority. Furthermore, the proposed framework is a learning-based method that is adaptable to various image modalities. Therefore, it has latent capacities to be integrated into surgical navigation systems.

Cortical surface registration using unsupervised learning

S3Reg: Superfast Spherical Surface Registration Based on Deep Learning

A Deep-Discrete Learning Framework for Spherical Surface Registration

SUGAR: Spherical ultrafast graph attention framework for cortical surface registration

Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces

Discriminative analysis of the human cortex using spherical CNNs - a study on Alzheimer's disease diagnosis

Reconstruction of Cortical Surfaces with Spherical Topology from Infant Brain MRI via Recurrent Deformation Learning

MsMorph: An Unsupervised pyramid learning network for brain image registration

GESH-Net: Graph-Enhanced Spherical Harmonic Convolutional Networks for Cortical Surface Registration

CortexMorph: fast cortical thickness estimation via diffeomorphic registration using VoxelMorph

Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration

JOSA: Joint surface-based registration and atlas construction of brain geometry and function

Unsupervised Multimodal Surface Registration with Geometric Deep Learning

Surface Fluid Registration of Conformal Representation: Application to Detect Disease Burden and Genetic Influence on Hippocampus

Real-Time 2D/3D Registration via CNN Regression and Centroid Alignment

DeepCSR: A 3D Deep Learning Approach for Cortical Surface Reconstruction

Self-Supervised Learning for Non-Rigid Registration Between Near-Isometric 3D Surfaces in Medical Imaging.

Towards Semantically-Consistent Deformable 2D-3D Registration for 3D Craniofacial Structure Estimation from A Single-View Lateral Cephalometric Radiograph

A novel brain registration model combining structural and functional MRI information

Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain