Abstract:Purpose: Ultrasound is often the preferred modality for image-guided therapy or treatment in organs such as liver due to real-time imaging capabilities. However, the reduced conspicuity of tumors in ultrasound images adversely impacts the precision and accuracy of treatment delivery. This problem is compounded by deformable motion due to breathing and other physiological activity. This creates the need for a fusion method to align interventional US with pre-interventional modalities that provide superior soft-tissue contrast (e.g., MRI) to accurately target a structure-of-interest and compensate for liver motion. Method: In this work, we developed a hybrid deformable fusion method to align 3D pre-interventional MRI and 3D interventional US volumes to target the structures-of-interest in liver accurately in real-time. The deformable multimodal fusion method involved an offline alignment of a pre-intervention MRI with a pre-intervention US volume using a traditional registration method, followed by real-time prediction of deformation using a trained deep-learning model between interventional US volumes across different respiratory states. This framework enables motion-compensated MRI-US image fusion in real-time for image-guided treatment. Results: The proposed hybrid deformable registration method was evaluated on three healthy volunteers across the pre-intervention MRI and 20 US volume pairs in the free-breathing respiratory cycle. The mean Euclidean landmark distance of three homologous targets in all three volunteers was less than 3 mm for percutaneous liver procedures. Conclusions: Preliminary results show that clinically acceptable registration accuracies for near real-time, deformable MRI-US fusion can be achieved by our proposed hybrid approach. The proposed combination of traditional and deep-learning deformable registration techniques is thus a promising approach for motion-compensated MRI-US fusion to improve targeting in image-guided liver interventions.

US & MR/CT Image Fusion with Markerless Skin Registration: A Proof of Concept

US \& MRI Image Fusion Based on Markerless Skin Registration

Image Fusion of CT and MR with Sparse Representation in NSST Domain

A hybrid deformable registration method to generate motion-compensated 3D virtual MRI for fusion with interventional real-time 3D ultrasound

3D ultrasound guided navigation system with hybrid image fusion

A Robust Registration Method for Real-Time Ultrasound Image Fusion with Pre-Acquired 3D Dataset

Registration Of 3d Fmt And Ct Images Of Mouse Via Affine Transformation Using Sequential Monte Carlo

MURF: Mutually Reinforcing Multi-Modal Image Registration and Fusion

Ultrasound and Fluoroscopic Images Fusion by Autonomous Ultrasound Probe Detection

Ultrasound Frame-to-Volume Registration via Deep Learning for Interventional Guidance

Unsupervised Fusion of Misaligned PAT and MRI Images via Mutually Reinforcing Cross-Modality Image Generation and Registration

HoloPOCUS: Portable Mixed-Reality 3D Ultrasound Tracking, Reconstruction and Overlay

Robotic Arm Based Automatic Ultrasound Scanning for Three-Dimensional Imaging

Freehand 3D Ultrasound Imaging Based on Probe-mounted Vision and IMU System

Freehand 2D Ultrasound Probe Calibration for Image Fusion with 3D MRI/CT

Three dimensional image alignment, registration and fusion

Three-dimensional multimodal medical imaging system based on free-hand ultrasound and structured light

Multitask Weakly Supervised Generative Network for MR-US Registration

A Novel Augmented Reality Ultrasound Framework Using an RGB-D Camera and a 3D-printed Marker

An application of multimodal image registration and fusion in a 3D tumor simulation model

A deep learning approach for 2D ultrasound and 3D CT/MR image registration in liver tumor ablation