Abstract:BACKGROUND: We applied a new concept of "image overlay surgery" consisting of the integration of virtual reality (VR) and augmented reality (AR) technology, in which dynamic 3D images were superimposed on the patient's actual body surface and evaluated as a reference for surgical navigation in gastrointestinal, hepatobiliary and pancreatic surgery.METHODS: We carried out seven surgeries, including three cholecystectomies, two gastrectomies and two colectomies. A Macintosh and a DICOM workstation OsiriX were used in the operating room for image analysis. Raw data of the preoperative patient information obtained via MDCT were reconstructed to volume rendering and projected onto the patient's body surface during the surgeries. For accurate registration, OsiriX was first set to reproduce the patient body surface, and the positional coordinates of the umbilicus, left and right nipples, and the inguinal region were fixed as physiological markers on the body surface to reduce the positional error.RESULTS: The registration process was non-invasive and markerlesss, and was completed within 5 min. Image overlay navigation was helpful for 3D anatomical understanding of the surgical target in the gastrointestinal, hepatobiliary and pancreatic anatomies. The surgeon was able to minimize movement of the gaze and could utilize the image assistance without interfering with the forceps operation, reducing the gap from the VR. Unexpected organ injury could be avoided in all procedures. In biliary surgery, the projected virtual cholangiogram on the abdominal wall could advance safely with identification of the bile duct. For early gastric and colorectal cancer, the small tumors and blood vessels, which usually could not be found on the gastric serosa by laparoscopic view, were simultaneously detected on the body surface by carbon dioxide-enhanced MDCT. This provided accurate reconstructions of the tumor and involved lymph node, directly linked with optimization of the surgical procedures.CONCLUSIONS: Our non-invasive markerless registration using physiological markers on the body surface reduced logistical efforts. The image overlay technique is a useful tool when highlighting hidden structures, giving more information.

Live demonstration of surgery across international borders with uncompressed high-definition quality

One hundred case studies of Asia-Pacific telemedicine using a digital video transport system over a research and education network

[Progress in imaging techniques]

Tele-proctoring for minimally invasive surgery across Japan: An initial step toward a new approach to improving the disparity of surgical care and supporting surgical education

Untethered and HIPAA-compliant Interactive Livestreaming of Surgery to Residents and Medical Students

A remote surgery experiment between Japan and Thailand over Internet using a low latency CODEC system

Application of international videoconferences for continuing medical education programs related to laparoscopic surgery.

Live-Streaming Surgery for Medical Student Education - Educational Solutions in Neurosurgery During the COVID-19 Pandemic

Interactive live-stream surgery contributes to surgical education in the context of contact restrictions

Micro- and macro-borderless surgery using a newly developed high-resolution (4K) three-dimensional video system

Immersive Live Streaming of Surgery Using 360-Degree Video to Head-Mounted Virtual Reality Devices: A New Paradigm in Surgical Education

International Internet2 Connectivity and Performance in Medical Imaging Applications: Bridging the Americas to Asia

[The experimental study on teleradiology system using HDTV and ISDN]

A multicenter survey of effects and challenges of an 8K ultra‐high‐definition endoscopy system compared to existing endoscopy systems for endoscopic surgery

[Laparoscopic adrenalectomy using telementoring system]

Surgical Video Projection onto a Mannequin: An Educational Tool for Simulation Practice of Perioperative Anesthetic Management

Triple-console robotic telesurgery: first impressions and future impact

Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery

Live-streaming otolaryngology surgical procedures for virtual medical student rotations

Educating Future Generations of Surgeons across Borders: Novel Global Linked Hybrid Live Cadaveric Peripheral Nerve Surgical Training Course

Maxillofacial Surgery 5.0: a new paradigm in telemedicine for distance surgery, remote assistance, and webinars