Abstract:Colonoscopy is the choice procedure to diagnose, screening, and treat the colon and rectum cancer, from early detection of small precancerous lesions (polyps), to confirmation of malign masses. However, the high variability of the organ appearance and the complex shape of both the colon wall and structures of interest make this exploration difficult. Learned visuospatial and perceptual abilities mitigate technical limitations in clinical practice by proper estimation of the intestinal depth. This work introduces a novel methodology to estimate colon depth maps in single frames from monocular colonoscopy videos. The generated depth map is inferred from the shading variation of the colon wall with respect to the light source, as learned from a realistic synthetic database. Briefly, a classic convolutional neural network architecture is trained from scratch to estimate the depth map, improving sharp depth estimations in haustral folds and polyps by a custom loss function that minimizes the estimation error in edges and curvatures. The network was trained by a custom synthetic colonoscopy database herein constructed and released, composed of 248400 frames (47 videos), with depth annotations at the level of pixels. This collection comprehends 5 subsets of videos with progressively higher levels of visual complexity. Evaluation of the depth estimation with the synthetic database reached a threshold accuracy of 95.65%, and a mean-RMSE of 0.451cm, while a qualitative assessment with a real database showed consistent depth estimations, visually evaluated by the expert gastroenterologist coauthoring this paper. Finally, the method achieved competitive performance with respect to another state-of-the-art method using a public synthetic database and comparable results in a set of images with other five state-of-the-art methods. Additionally, three-dimensional reconstructions demonstrated useful approximations of the gastrointestinal tract geometry. Code for reproducing the reported results and the dataset are available at https://github.com/Cimalab-unal/ColonDepthEstimation.

Deep learning-based image exposure enhancement as a pre-processing for an accurate 3D colon surface reconstruction

Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging

High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces

Lighting Enhancement Aids Reconstruction of Colonoscopic Surfaces

Sparse-to-dense coarse-to-fine depth estimation for colonoscopy

An Educated Warm Start For Deep Image Prior-Based Micro CT Reconstruction

RNNSLAM: Reconstructing the 3D colon to visualize missing regions during a colonoscopy

Deep Learning and Conditional Random Fields-based Depth Estimation and Topographical Reconstruction from Conventional Endoscopy

Leveraging a realistic synthetic database to learn Shape-from-Shading for estimating the colon depth in colonoscopy images

Photometric single-view dense 3D reconstruction in endoscopy

Self-supervised monocular depth estimation for high field of view colonoscopy cameras

A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Models

Deep Learning-Based Reconstruction of Interventional Tools from Four X-Ray Projections for Tomographic Interventional Guidance

ColonNeRF: High-Fidelity Neural Reconstruction of Long Colonoscopy

Diagnostic evaluation of a deep learning model for optical diagnosis of colorectal cancer

SLAM Endoscopy enhanced by adversarial depth prediction

SimCol3D -- 3D Reconstruction during Colonoscopy Challenge

Deep learning for gastroscopic images: computer-aided techniques for clinicians

Deep Learning-based Biological Anatomical Landmark Detection in Colonoscopy Videos

3D reconstruction from endoscopy images: A survey

Deep Learning Angiography (DLA): Three-dimensional C-arm Cone Beam CT Angiography Using Deep Learning