Abstract:Background: The fusion of computed tomography (CT) and ultrasound (US) image can enhance lesion detection ability and improve the success rate of liver interventional radiology. The image-based fusion methods encounter the challenge of registration initialization due to the random scanning pose and limited field of view of US. Existing automatic methods those used vessel geometric information and intensity-based metric are sensitive to parameters and have low success rate. The learning-based methods require a large number of registered datasets for training. Purpose: The aim of this study is to provide a fully automatic and robust US-3D CT registration method without registered training data and user-specified parameters assisted by the revolutionary deep learning-based segmentation, which can further be used for preparing training samples for the study of learning-based methods. Methods: We propose a fully automatic CT-3D US registration method by two improved registration metrics. We propose to use 3D U-Net-based multi-organ segmentation of US and CT to assist the conventional registration. The rigid transform is searched in the space of any paired vessel bifurcation planes where the best transform is decided by a segmentation overlap metric, which is more related to the segmentation precision than Dice coefficient. In nonrigid registration phase, we propose a hybrid context and edge based image similarity metric with a simple mask that can remove most noisy US voxels to guide the B-spline transform registration. We evaluate our method on 42 paired CT-3D US datasets scanned with two different US devices from two hospitals. We compared our methods with other exsiting methods with both quantitative measures of target registration error (TRE) and the Jacobian determinent with paired t-test and qualitative registration imaging results. Results: The results show that our method achieves fully automatic rigid registration TRE of 4.895 mm, deformable registration TRE of 2.995 mm in average, which outperforms state-of-the-art automatic linear methods and nonlinear registration metrics with paired t-test's p value less than 0.05. The proposed overlap metric achieves better results than self similarity description (SSD), edge matching (EM), and block matching (BM) with p values of 1.624E-10, 4.235E-9, and 0.002, respectively. The proposed hybrid edge and context-based metric outperforms context-only, edge-only, and intensity statistics-only-based metrics with p values of 0.023, 3.81E-5, and 1.38E-15, respectively. The 3D US segmentation has achieved mean Dice similarity coefficient (DSC) of 0.799, 0.724, 0.788, and precision of 0.871, 0.769, 0.862 for gallbladder, vessel, and branch vessel, respectively. Conclusions: The deep learning-based US segmentation can achieve satisfied result to assist robust conventional rigid registration. The Dice similarity coefficient-based metrics, hybrid context, and edge image similarity metric contribute to robust and accurate registration.

Deformable registration of chest CT images using a 3D convolutional neural network based on unsupervised learning

Learning 3D Non-Rigid Deformation Based on an Unsupervised Deep Learning for PET/CT Image Registration

A FCN-based Unsupervised Learning Model for Deformable Chest CT Image Registration

An Unsupervised Deep Learning Framework for Large-Scale Lung CT Deformable Image Registration

Unsupervised End-to-end Learning for Deformable Medical Image Registration

Respiratory Deformation Registration in 4D-Ct/cone Beam CT Using Deep Learning.

Residual Fully Convolutional Deformable Registration Network for Deformable Lung CT Images

An Optimized Registration Method Based on Distribution Similarity and DVF Smoothness for 3D PET and CT Images

Unsupervised Lung CT Image Registration Via Stochastic Decomposition of Deformation Fields

Deformable Lung CT Registration by Decomposing Large Deformation

3D deformable registration of longitudinal abdominopelvic CT images using unsupervised deep learning

Deep-Learning-based Fast and Accurate 3D CT Deformable Image Registration in Lung Cancer

CNN-based Lung CT Registration with Multiple Anatomical Constraints

CCMNet: Cross-scale correlation-aware mapping network for 3D lung CT image registration

A robust and automatic CT-3D ultrasound registration method based on segmentation, context, and edge hybrid metric

Unsupervised learning for deformable registration of thoracic CT and cone‐beam CT based on multiscale features matching with spatially adaptive weighting

A New Unsupervised Learning Method for 3D Deformable Medical Image Registration

Deform U-Net: Unsupervised Deformable 3D Biomedical Image Registration Network

An Unsupervised Learning-Based Multi-Organ Registration Method for 3D Abdominal CT Images

An unsupervised multi-scale framework with attention-based network (MANet) for lung 4D-CT registration

Deformable Image Registration of Chest CT Using Deep Cycle Network.