Abstract:Purpose: Accurate deformable registration of magnetic resonance imaging (MRI) scans containing pathologies is challenging due to changes in tissue appearance. In this paper, we developed a novel automated three-dimensional (3D) convolutional U-Net based deformable image registration (ConvUNet-DIR) method using unsupervised learning to establish correspondence between baseline pre-operative and follow-up MRI scans of patients with brain glioma. Methods: This study involved multi-parametric brain MRI scans (T1, T1-contrast enhanced, T2, FLAIR) acquired at pre-operative and follow-up time for 160 patients diagnosed with glioma, representing the BraTS-Reg 2022 challenge dataset. ConvUNet-DIR, a deep learning-based deformable registration workflow using 3D U-Net style architecture as a core, was developed to establish correspondence between the MRI scans. The workflow consists of three components: (1) the U-Net learns features from pairs of MRI scans and estimates a mapping between them, (2) the grid generator computes the sampling grid based on the derived transformation parameters, and (3) the spatial transformation layer generates a warped image by applying the sampling operation using interpolation. A similarity measure was used as a loss function for the network with a regularization parameter limiting the deformation. The model was trained via unsupervised learning using pairs of MRI scans on a training data set (n = 102) and validated on a validation data set (n = 26) to assess its generalizability. Its performance was evaluated on a test set (n = 32) by computing the Dice score and structural similarity index (SSIM) quantitative metrics. The model's performance also was compared with the baseline state-of-the-art VoxelMorph (VM1 and VM2) learning-based algorithms. Results: The ConvUNet-DIR model showed promising competency in performing accurate 3D deformable registration. It achieved a mean Dice score of 0.975 ± 0.003 and SSIM of 0.908 ± 0.011 on the test set (n = 32). Experimental results also demonstrated that ConvUNet-DIR outperformed the VoxelMorph algorithms concerning Dice (VM1: 0.969 ± 0.006 and VM2: 0.957 ± 0.008) and SSIM (VM1: 0.893 ± 0.012 and VM2: 0.857 ± 0.017) metrics. The time required to perform a registration for a pair of MRI scans is about 1 s on the CPU. Conclusions: The developed deep learning-based model can perform an end-to-end deformable registration of a pair of 3D MRI scans for glioma patients without human intervention. The model could provide accurate, efficient, and robust deformable registration without needing pre-alignment and labeling. It outperformed the state-of-the-art VoxelMorph learning-based deformable registration algorithms and other supervised/unsupervised deep learning-based methods reported in the literature.

Dense Error Map Estimation for MRI-Ultrasound Registration in Brain Tumor Surgery Using Swin UNETR

FocalErrorNet: Uncertainty-aware focal modulation network for inter-modal registration error estimation in ultrasound-guided neurosurgery

Toward Estimating MRI-Ultrasound Registration Error in Image-Guided Neurosurgery

Non-rigid registration of 3D ultrasound for neurosurgery using automatic feature detection and matching

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

Evaluation of MRI to Ultrasound Registration Methods for Brain Shift Correction: The CuRIOUS2018 Challenge

Automatic brain tumor segmentation in 2D intra-operative ultrasound images using MRI tumor annotations

Unsupervised Deformable Image Registration with Absent Correspondences in Pre-operative and Post-Recurrence Brain Tumor MRI Scans

Deep learning-based deformable image registration with bilateral pyramid to align pre-operative and follow-up magnetic resonance imaging (MRI) scans

Label-driven magnetic resonance imaging (MRI)-transrectal ultrasound (TRUS) registration using weakly supervised learning for MRI-guided prostate radiotherapy

A deep learning approach for 2D ultrasound and 3D CT/MR image registration in liver tumor ablation

MDH-Net: Advancing 3D Brain MRI Registration with Multi-Stage Transformer and Dual-Stream Feature Refinement Hybrid Network

2D/3D US-TO-MRI RIGID REGISTRATION BY DEEP LEARNING

Unsupervised Deformable Image Registration Network for 3D Medical Images

A NOVEL DEEP LEARNING METHOD FOR BRAIN TUMOR SEGMENTATION IN MAGNETIC RESONANCE IMAGES BASED ON RESIDUAL UNITS AND MODIFIED U-NET MODEL

Deformable MRI-US Image Registration Using Biomechanically Constrained Shape Matching Network for Image-Guided Prostate Intervention

Robust and efficient 3D registration via depth map-based feature point matching in image-guided neurosurgery

Deformable MRI-TRUS Registration Using Biomechanically Constrained Deep Learning Model for Tumor-Targeted Prostate Brachytherapy

A Deep Network for Joint Registration and Reconstruction of Images with Pathologies

Mri-Us Registration Using Label-Driven Weakly-Supervised Learning For Multiparametric Mri-Guided Hdr Prostate Brachytherapy

Tissue characterization based on the analysis on i3DUS data for diagnosis support in neurosurgery