Abstract:Endoscopic surgery relies on two-dimensional views, posing challenges for surgeons in depth perception and instrument manipulation. While Simultaneous Localization and Mapping (SLAM) has emerged as a promising solution to address these limitations, its implementation in endoscopic procedures presents significant challenges due to hardware limitations, such as the use of a monocular camera and the absence of odometry sensors. This study presents a robust deep learning-based SLAM approach that combines state-of-the-art and newly developed models. It consists of three main parts: the Monocular Pose Estimation Module that introduces a novel unsupervised method based on the CycleGAN architecture, the Monocular Depth Estimation Module that leverages the novel Zoe architecture, and the 3D Reconstruction Module which uses information from the previous models to create a coherent surgical map. The performance of the procedure was rigorously evaluated using three publicly available datasets (Hamlyn, EndoSLAM, and SCARED) and benchmarked against two state-of-the-art methods, EndoSFMLearner and EndoDepth. The integration of Zoe in the MDEM demonstrated superior performance compared to state-of-the-art depth estimation algorithms in endoscopy, whereas the novel approach in the MPEM exhibited competitive performance and the lowest inference time. The results showcase the robustness of our approach in laparoscopy, gastroscopy, and colonoscopy, three different scenarios in endoscopic surgery. The proposed SLAM approach has the potential to improve the accuracy and efficiency of endoscopic procedures by providing surgeons with enhanced depth perception and 3D reconstruction capabilities.

DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences

NR-SLAM: Non-Rigid Monocular SLAM

Tracking monocular camera pose and deformation for SLAM inside the human body

Direct and Sparse Deformable Tracking

Real-time deformable SLAM with geometrically adapted template for dynamic monocular laparoscopic scenes

Monocular SLAM for Large Scale Scenes

An observable time series based SLAM algorithm for deforming environment

Robust Monocular SLAM in Dynamic Environments

Local Optimized and Scalable Frame-to-model SLAM

MIS-SLAM: Real-time Large Scale Dense Deformable SLAM System in Minimal Invasive Surgery Based on Heterogeneous Computing

EMDQ-SLAM: Real-Time High-Resolution Reconstruction of Soft Tissue Surface from Stereo Laparoscopy Videos

LSD-SLAM: Large-Scale Direct Monocular SLAM

MonoSLAM: Real-Time Single Camera SLAM

BodySLAM: A Generalized Monocular Visual SLAM Framework for Surgical Applications

Dynamic Reconstruction of Deformable Soft-tissue with Stereo Scope in Minimal Invasive Surgery

ENeRF-SLAM:A Dense Endoscopic SLAM with Neural Implicit Representation

Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM

HFN-SLAM:Hybrid Scene Neural Representation SLAM Based on Frame Alignment and Normal Consistency

Event-Based Line SLAM in Real-Time

High-resolution Ecosystem Mapping in Repetitive Environments Using Dual Camera SLAM

CD-SLAM:A Real-Time Stereo Visual-Inertial SLAM for Complex Dynamic Environments with Semantic and Geometric Information