Abstract:Quantitative calculations of camera poses and dense anatomical reconstructions from endoscopic videos are essential for applications such as intraoperative navigation and robotic surgery automation. Prior studies on this task either overlook the unique characteristics of endoscopic scenes or produce reconstructions with numerous gaps due to limited observations, significantly limiting their practical application. Inspired by recent advancements in neural rendering, we develop a dense visual SLAM system that employs neural implicit representations, specifically designed for endoscopic sequences. By incorporating 3D geometric scene priors, our system effectively predicts and fills in unseen areas, ensuring the continuous and complete reconstruction of the scene. Taking into account the dynamic nature of the light source and the confined anatomy of the human body, we propose a neural implicit representation method designed for endoscopic scenes. Additionally, we introduce a hybrid tracking method that merges Gauss-Newton and gradient-based pose optimization, improving geometric consistency across frames. This reduces reliance on precise data matching and significantly enhances camera tracking accuracy. Extensive experiments on both synthetic and real medical endoscopic datasets demonstrate that our method outperforms existing systems in terms of scene reconstruction quality, camera tracking accuracy, and image rendering quality. Code is available at: https://github.com/Mar-lll/ENeRF-SLAM.

An Algorithm for the SE(3)-Transformation on Neural Implicit Maps for Remapping Functions

SDFMAP: Neural Signed Distance Fields for Mapping and Positioning in Real-Time

Neural Surfel Reconstruction: Addressing Loop Closure Challenges in Large-Scale 3D Neural Scene Mapping

Efficient Map Fusion for Multiple Implicit SLAM Agents

LISNeRF Mapping: LiDAR-based Implicit Mapping via Semantic Neural Fields for Large-Scale 3D Scenes

S3-SLAM: Sparse Tri-plane Encoding for Neural Implicit SLAM

MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction

ENeRF-SLAM:A Dense Endoscopic SLAM with Neural Implicit Representation

NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction With Implicit Neural Representations

SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and Quasi-Planar Segmentation

Swift-Mapping: Online Neural Implicit Dense Mapping in Urban Scenes

Self-Calibrated Neural Implicit 3D Reconstruction

Towards Large-Scale Incremental Dense Mapping using Robot-centric Implicit Neural Representation

Efficient Implicit Neural Reconstruction Using LiDAR

Efficient 3D Instance Mapping and Localization with Neural Fields

HI-SLAM: Monocular Real-time Dense Mapping with Hybrid Implicit Fields

SNI-SLAM: Semantic Neural Implicit SLAM

Structerf-SLAM: Neural implicit representation SLAM for structural environments

3D LiDAR Mapping in Dynamic Environments Using a 4D Implicit Neural Representation

EMIE-MAP: Large-Scale Road Surface Reconstruction Based on Explicit Mesh and Implicit Encoding

NIS-SLAM: Neural Implicit Semantic RGB-D SLAM for 3D Consistent Scene Understanding