Abstract:Purpose: Ultrasound is often the preferred modality for image-guided therapy or treatment in organs such as liver due to real-time imaging capabilities. However, the reduced conspicuity of tumors in ultrasound images adversely impacts the precision and accuracy of treatment delivery. This problem is compounded by deformable motion due to breathing and other physiological activity. This creates the need for a fusion method to align interventional US with pre-interventional modalities that provide superior soft-tissue contrast (e.g., MRI) to accurately target a structure-of-interest and compensate for liver motion. Method: In this work, we developed a hybrid deformable fusion method to align 3D pre-interventional MRI and 3D interventional US volumes to target the structures-of-interest in liver accurately in real-time. The deformable multimodal fusion method involved an offline alignment of a pre-intervention MRI with a pre-intervention US volume using a traditional registration method, followed by real-time prediction of deformation using a trained deep-learning model between interventional US volumes across different respiratory states. This framework enables motion-compensated MRI-US image fusion in real-time for image-guided treatment. Results: The proposed hybrid deformable registration method was evaluated on three healthy volunteers across the pre-intervention MRI and 20 US volume pairs in the free-breathing respiratory cycle. The mean Euclidean landmark distance of three homologous targets in all three volunteers was less than 3 mm for percutaneous liver procedures. Conclusions: Preliminary results show that clinically acceptable registration accuracies for near real-time, deformable MRI-US fusion can be achieved by our proposed hybrid approach. The proposed combination of traditional and deep-learning deformable registration techniques is thus a promising approach for motion-compensated MRI-US fusion to improve targeting in image-guided liver interventions.

Ultrasound and Fluoroscopic Images Fusion by Autonomous Ultrasound Probe Detection

Direct 3D ultrasound fusion for transesophageal echocardiography

Ultrasound Frame-to-Volume Registration via Deep Learning for Interventional Guidance

Registration of 3D trans-esophageal echocardiography to X-ray fluoroscopy using image-based probe tracking

Automatic multi-view pose estimation in focused cardiac ultrasound

Freehand 2D Ultrasound Probe Calibration for Image Fusion with 3D MRI/CT

Freehand 3D Ultrasound Imaging Based on Probe-mounted Vision and IMU System

Multi-Modal Autonomous Ultrasound Scanning for Efficient Human–Machine Fusion Interaction

Enhancing Multi-Object Detection in Ultrasound Images Through Semi-Supervised Learning, Focal Loss and Relation of Frame

Towards Autonomous Robotic Minimally Invasive Ultrasound Scanning and Vessel Reconstruction on Non-Planar Surfaces

A hybrid deformable registration method to generate motion-compensated 3D virtual MRI for fusion with interventional real-time 3D ultrasound

Domain Knowledge-Guided Adversarial Adaptive Fusion of Hybrid Breast Ultrasound Data

EchoFusion: Tracking and Reconstruction of Objects in 4D Freehand Ultrasound Imaging without External Trackers

3D ultrasound guided navigation system with hybrid image fusion

Gpu-Based Fast 3d Ultrasound-Endoscope Image Fusion For Complex-Shaped Objects

Autonomous control of an ultrasound probe for intra-operative ultrasonography using vision-based shape sensing of pneumatically attachable flexible rails

Six-Degree-of-Freedom Freehand 3D Ultrasound: A Low-Cost Computer Vision-Based Approach for Orthopedic Applications

US & MR/CT Image Fusion with Markerless Skin Registration: A Proof of Concept

Patch-based field-of-view matching in multi-modal images for electroporation-based ablations

Image fusion on the endoscopic view for endo-nasal skull-base surgery.

FMB: Dual-view fusion and registration of 2D DSA images and 3D MRA images for neurointerventional-based procedures