Abstract:Purpose: Ultrasound is often the preferred modality for image-guided therapy or treatment in organs such as liver due to real-time imaging capabilities. However, the reduced conspicuity of tumors in ultrasound images adversely impacts the precision and accuracy of treatment delivery. This problem is compounded by deformable motion due to breathing and other physiological activity. This creates the need for a fusion method to align interventional US with pre-interventional modalities that provide superior soft-tissue contrast (e.g., MRI) to accurately target a structure-of-interest and compensate for liver motion. Method: In this work, we developed a hybrid deformable fusion method to align 3D pre-interventional MRI and 3D interventional US volumes to target the structures-of-interest in liver accurately in real-time. The deformable multimodal fusion method involved an offline alignment of a pre-intervention MRI with a pre-intervention US volume using a traditional registration method, followed by real-time prediction of deformation using a trained deep-learning model between interventional US volumes across different respiratory states. This framework enables motion-compensated MRI-US image fusion in real-time for image-guided treatment. Results: The proposed hybrid deformable registration method was evaluated on three healthy volunteers across the pre-intervention MRI and 20 US volume pairs in the free-breathing respiratory cycle. The mean Euclidean landmark distance of three homologous targets in all three volunteers was less than 3 mm for percutaneous liver procedures. Conclusions: Preliminary results show that clinically acceptable registration accuracies for near real-time, deformable MRI-US fusion can be achieved by our proposed hybrid approach. The proposed combination of traditional and deep-learning deformable registration techniques is thus a promising approach for motion-compensated MRI-US fusion to improve targeting in image-guided liver interventions.

Real-Time Deformable Registration for Robotic Ultrasound Surgical Guidance Based on Vectorized Trust-Region Optimization

Ultrasound-Guided Robotic Autonomous Operation Based on Real-Time Deformation Tracking and Prediction

Robust Orthogonal-View 2-D/3-D Rigid Registration for Minimally Invasive Surgery

Non-rigid Registration Algorithm Based on Modally Controlled Free Form Deformation

Medical Image Registration Algorithm in Image-Guided Noninvasive Surgery

Ultrasound Image Guided and Mixed Reality-Based Surgical System With Real-Time Soft Tissue Deformation Computing for Robotic Cervical Pedicle Screw Placement

Precise Repositioning of Robotic Ultrasound: Improving Registration-Based Motion Compensation Using Ultrasound Confidence Optimization

Fast Registration of Intraoperative Ultrasound and Preoperative MR Images Based on Calibrations of 2D and 3D Ultrasound Probes

Real-to-Sim Registration of Deformable Soft Tissue with Position-Based Dynamics for Surgical Robot Autonomy

A Real-time and Registration-free Framework for Dynamic Shape Instantiation

Non-rigid registration of 3D ultrasound for neurosurgery using automatic feature detection and matching

Ultrasound Frame-to-Volume Registration via Deep Learning for Interventional Guidance

Automatic Bone Segmentation and Ultrasound—CT Registration for Robotic Assisted Femoral Shaft Fracture Reduction

Real-time Prediction of Soft Tissue Deformations Using Data-driven Nonlinear Presurgical Simulations

A Robust Registration Algorithm for Image-Guided Surgical Robot

A hybrid deformable registration method to generate motion-compensated 3D virtual MRI for fusion with interventional real-time 3D ultrasound

Deformable Image Registration using Unsupervised Deep Learning for CBCT-guided Abdominal Radiotherapy

Inter-fraction Deformable Image Registration Using Unsupervised Deep Learning for CBCT-guided Abdominal Radiotherapy

A deep learning approach for 2D ultrasound and 3D CT/MR image registration in liver tumor ablation

Arc-to-line frame registration method for ultrasound and photoacoustic image-guided intraoperative robot-assisted laparoscopic prostatectomy

An Image Registration Method for Surgical Robots Based on Human-Robot Cooperation