Abstract:During minimal invasive surgery (MIS), the laparoscope only provides a single viewpoint to the surgeon, leaving a lack of 3D perception. Many works have been proposed to obtain depth and 3D reconstruction by designing a new optical structure or by depending on the camera pose and image sequences. Most of these works modify the structure of the conventional laparoscopes and cannot provide 3D reconstruction of different magnification views. In this study, we propose a laparoscopic system based on double liquid lenses, which provide doctors with variable magnification rates, near observation, and real-time monocular 3D reconstruction. Our system composes of an optical structure that can obtain auto magnification change and autofocus without any physically moving element, and a deep learning network based on the Depth from Defocus (DFD) method, trained to suit inconsistent camera intrinsic situations and estimate depth from images of different focal lengths. The optical structure is portable and can be mounted on conventional laparoscopes. The depth estimation network estimates depth in real-time from monocular images of different focal lengths and magnification rates. Experiments show that our system provides a 0.68-1.44x zoom rate and can estimate depth from different magnification rates at 6fps. Monocular 3D reconstruction reaches at least 6mm accuracy. The system also provides a clear view even under 1mm close working distance. Ex-vivo experiments and implementation on clinical images prove that our system provides doctors with a magnified clear view of the lesion, as well as quick monocular depth perception during laparoscopy, which help surgeons get better detection and size diagnosis of the abdomen during laparoscope surgeries.

A Foldable Stereo Vision Unit for Single Port Access Laparoscopy

Linkage Couples Based Endoscope-holder Robot with Enhanced Bending Curvature for Laparoscopic Surgery

Design of the SJTU Unfoldable Robotic System (SURS) for Single Port Laparoscopy

Design, simulation and evaluation of kinematic alternatives for Insertable Robotic Effectors Platforms in Single Port Access Surgery

Design of a Robotic Laparoscopic Tool with Modular Actuation.

A Robotic Laparoscopic Tool with Enhanced Capabilities and Modular Actuation

Vision-Based Real-Time Tracking of Surgical Instruments in Robot-Assisted Laparoscopic Surgery

System Design of an Insertable Robotic Effector Platform for Single Port Access (SPA) Surgery

Mechanical design of wireless in vivo robot unit for surgical vision

Development of the SJTU Unfoldable Robotic System (SURS) for Single Port Laparoscopy

Design and Coordination Kinematics of an Insertable Robotic Effectors Platform for Single-Port Access Surgery

Development of miniaturized light endoscope-holder robot for laparoscopic surgery

Dexterity and Functionality Enhancement of the SJTU Unfoldable Robotic System.

A Time-of-Flight Stereoscopic Endoscope for Anatomical 3D Reconstruction

Surgical robots for SPL and NOTES: a review.

Robotic Manipulator‐Assisted Omnidirectional Augmented Reality for Endoluminal Intervention Telepresence

A Monocular Variable Magnifications 3D Laparoscope System Using Double Liquid Lenses

Configuration Comparison For Surgical Robotic Systems Using A Single Access Port And Continuum Mechanisms

Gaze Contingent Control for an Articulated Mechatronic Laparoscope

A Novel Handheld Bimanual&nbsp;Surgical Robot for&nbsp;Single-Port Laparoscopic Surgery

Intraoperative laparoscopic photoacoustic image guidance system in the da Vinci surgical system