Abstract:Colonoscopy is the choice procedure to diagnose, screening, and treat the colon and rectum cancer, from early detection of small precancerous lesions (polyps), to confirmation of malign masses. However, the high variability of the organ appearance and the complex shape of both the colon wall and structures of interest make this exploration difficult. Learned visuospatial and perceptual abilities mitigate technical limitations in clinical practice by proper estimation of the intestinal depth. This work introduces a novel methodology to estimate colon depth maps in single frames from monocular colonoscopy videos. The generated depth map is inferred from the shading variation of the colon wall with respect to the light source, as learned from a realistic synthetic database. Briefly, a classic convolutional neural network architecture is trained from scratch to estimate the depth map, improving sharp depth estimations in haustral folds and polyps by a custom loss function that minimizes the estimation error in edges and curvatures. The network was trained by a custom synthetic colonoscopy database herein constructed and released, composed of 248400 frames (47 videos), with depth annotations at the level of pixels. This collection comprehends 5 subsets of videos with progressively higher levels of visual complexity. Evaluation of the depth estimation with the synthetic database reached a threshold accuracy of 95.65%, and a mean-RMSE of 0.451cm, while a qualitative assessment with a real database showed consistent depth estimations, visually evaluated by the expert gastroenterologist coauthoring this paper. Finally, the method achieved competitive performance with respect to another state-of-the-art method using a public synthetic database and comparable results in a set of images with other five state-of-the-art methods. Additionally, three-dimensional reconstructions demonstrated useful approximations of the gastrointestinal tract geometry. Code for reproducing the reported results and the dataset are available at https://github.com/Cimalab-unal/ColonDepthEstimation.

Delay in the diagnosis and treatment of pulmonary tuberculosis in patients attending a university teaching hospital.

Colonoscopy Coverage Revisited: Identifying Scanning Gaps in Real-Time

Sparse-to-dense coarse-to-fine depth estimation for colonoscopy

ColonNeRF: High-Fidelity Neural Reconstruction of Long Colonoscopy

VISUALIZING MISSING SURFACES IN COLONOSCOPY VIDEOS USING SHARED LATENT SPACE REPRESENTATIONS

SimCol3D -- 3D Reconstruction during Colonoscopy Challenge

Motion-based Camera Localization System in Colonoscopy Videos

High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces

SLAM Endoscopy enhanced by adversarial depth prediction

Estimating the coverage in 3d reconstructions of the colon from colonoscopy videos

Disturbances of sleep by noise

Lighting Enhancement Aids Reconstruction of Colonoscopic Surfaces

Topological SLAM in colonoscopies leveraging deep features and topological priors

Deep Learning-based Biological Anatomical Landmark Detection in Colonoscopy Videos

ColNav: Real-Time Colon Navigation for Colonoscopy

Photometric single-view dense 3D reconstruction in endoscopy

Gaussian Pancakes: Geometrically-Regularized 3D Gaussian Splatting for Realistic Endoscopic Reconstruction

Stereo Dense Scene Reconstruction and Accurate Localization for Learning-Based Navigation of Laparoscope in Minimally Invasive Surgery

3D Reconstruction of the Human Colon from Capsule Endoscope Video

Leveraging a realistic synthetic database to learn Shape-from-Shading for estimating the colon depth in colonoscopy images

Colonoscopy 3D video dataset with paired depth from 2D-3D registration