Abstract:BACKGROUND: We applied a new concept of "image overlay surgery" consisting of the integration of virtual reality (VR) and augmented reality (AR) technology, in which dynamic 3D images were superimposed on the patient's actual body surface and evaluated as a reference for surgical navigation in gastrointestinal, hepatobiliary and pancreatic surgery.METHODS: We carried out seven surgeries, including three cholecystectomies, two gastrectomies and two colectomies. A Macintosh and a DICOM workstation OsiriX were used in the operating room for image analysis. Raw data of the preoperative patient information obtained via MDCT were reconstructed to volume rendering and projected onto the patient's body surface during the surgeries. For accurate registration, OsiriX was first set to reproduce the patient body surface, and the positional coordinates of the umbilicus, left and right nipples, and the inguinal region were fixed as physiological markers on the body surface to reduce the positional error.RESULTS: The registration process was non-invasive and markerlesss, and was completed within 5 min. Image overlay navigation was helpful for 3D anatomical understanding of the surgical target in the gastrointestinal, hepatobiliary and pancreatic anatomies. The surgeon was able to minimize movement of the gaze and could utilize the image assistance without interfering with the forceps operation, reducing the gap from the VR. Unexpected organ injury could be avoided in all procedures. In biliary surgery, the projected virtual cholangiogram on the abdominal wall could advance safely with identification of the bile duct. For early gastric and colorectal cancer, the small tumors and blood vessels, which usually could not be found on the gastric serosa by laparoscopic view, were simultaneously detected on the body surface by carbon dioxide-enhanced MDCT. This provided accurate reconstructions of the tumor and involved lymph node, directly linked with optimization of the surgical procedures.CONCLUSIONS: Our non-invasive markerless registration using physiological markers on the body surface reduced logistical efforts. The image overlay technique is a useful tool when highlighting hidden structures, giving more information.

P-BIO-01 Development of LCD Slice Image Overlay Device with Electromagnetic Wave Measurement(Bio-medical Equipments,Technical Program of Poster Session)

Non-metal Slice Image Overlay Display System Used Inside the Open Type MRI

J1102-1-6 Evaluation of Influence on MRI Scans to Electromagnetic Waves for Open Type MRI

Endoscopic Optical Coherence Tomography Angiography Using Inverse SNR-amplitude Decorrelation Features and Electrothermal Micro-Electro-mechanical System Raster Scan

MR-compatible, 3.8 inch dual organic light-emitting diode (OLED) in-bore display for functional MRI

Surgical navigation of integral videography image overlay for open MRI-Guided glioma surgery

MRI compatibility study of an integrated PET/RF-coil prototype system at 3T

Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery

Three Dimensional Real-Time Image Overlay Navigation System Using Integral Videography In Knee-Joint Surgery: Evaluation Of Motion Tracking Capability

Adaptive Cylindrical Wireless Metasurfaces in Clinical Magnetic Resonance Imaging

Development of high resolution programmable oversampling MI sensor system with 32-bit ADC for multi-channel bio-magnetic measurements

Developing a Real-Time Photoacoustic-Ultrasound Dual Modality Functional Imaging System (conference Presentation)

Navigation technique for MR-endoscope system using a wireless accelerometer-based remote control device

Development of a Dual Tactile Pattern Presentation Device Using in MRI Environment

Miniature Magnetic Resonance Imaging System for in Situ Monitoring of Bacterial Growth and Biofilm Formation

Development of High-Speed Image Processing System for High Quality Integral Videography

An electroencephalogram microdisplay to visualize neuronal activity on the brain surface

Real-time passive cavitation mapping and B-mode fusion imaging via hybrid adaptive beamformer with modified diagnostic ultrasound platform

Intra-operative Real-Time 3-D Information Display System Based on Integral Videography

Toward Next-Generation Molecular Imaging with a Clinical Low-Field (0.064 T) Point-of-Care MRI Scanner

Enhancement of magnetic resonance imaging with metasurfaces