Abstract:Colonoscopy is the choice procedure to diagnose, screening, and treat the colon and rectum cancer, from early detection of small precancerous lesions (polyps), to confirmation of malign masses. However, the high variability of the organ appearance and the complex shape of both the colon wall and structures of interest make this exploration difficult. Learned visuospatial and perceptual abilities mitigate technical limitations in clinical practice by proper estimation of the intestinal depth. This work introduces a novel methodology to estimate colon depth maps in single frames from monocular colonoscopy videos. The generated depth map is inferred from the shading variation of the colon wall with respect to the light source, as learned from a realistic synthetic database. Briefly, a classic convolutional neural network architecture is trained from scratch to estimate the depth map, improving sharp depth estimations in haustral folds and polyps by a custom loss function that minimizes the estimation error in edges and curvatures. The network was trained by a custom synthetic colonoscopy database herein constructed and released, composed of 248400 frames (47 videos), with depth annotations at the level of pixels. This collection comprehends 5 subsets of videos with progressively higher levels of visual complexity. Evaluation of the depth estimation with the synthetic database reached a threshold accuracy of 95.65%, and a mean-RMSE of 0.451cm, while a qualitative assessment with a real database showed consistent depth estimations, visually evaluated by the expert gastroenterologist coauthoring this paper. Finally, the method achieved competitive performance with respect to another state-of-the-art method using a public synthetic database and comparable results in a set of images with other five state-of-the-art methods. Additionally, three-dimensional reconstructions demonstrated useful approximations of the gastrointestinal tract geometry. Code for reproducing the reported results and the dataset are available at https://github.com/Cimalab-unal/ColonDepthEstimation.

Colonoscopy Coverage Revisited: Identifying Scanning Gaps in Real-Time

Estimating the coverage in 3d reconstructions of the colon from colonoscopy videos

Detecting Deficient Coverage in Colonoscopies

RNNSLAM: Reconstructing the 3D colon to visualize missing regions during a colonoscopy

VISUALIZING MISSING SURFACES IN COLONOSCOPY VIDEOS USING SHARED LATENT SPACE REPRESENTATIONS

Colonoscopy 3D video dataset with paired depth from 2D-3D registration

ColNav: Real-Time Colon Navigation for Colonoscopy

Novel polyp detection technology for colonoscopy: 3D optical scanner

Artificial intelligence to improve polyp detection and screening time in colon capsule endoscopy

Sa1930 MISSED COLONIC SURFACE AREA AT COLONOSCOPY CAN BE CALCULATED WITH COMPUTERIZED 3D RECONSTRUCTION

SimCol3D -- 3D Reconstruction during Colonoscopy Challenge

3D Reconstruction of the Human Colon from Capsule Endoscope Video

Sparse-to-dense coarse-to-fine depth estimation for colonoscopy

An Image Segmentation Approach to Extract Colon Lumen Through Colonic Material Tagging and Hidden Markov Random Field Model for Virtual Colonoscopy

Fully Automated Colon Segmentation for the Computation of Complete Colon Centerline in Virtual Colonoscopy

Segmentation of colon and removal of opacified fluid for virtual colonoscopy

Self-supervised monocular depth estimation for high field of view colonoscopy cameras

Real-time automatic polyp detection in colonoscopy using feature enhancement module and spatiotemporal similarity correlation unit

SegCol Challenge: Semantic Segmentation for Tools and Fold Edges in Colonoscopy data

Motion-based Camera Localization System in Colonoscopy Videos

Leveraging a realistic synthetic database to learn Shape-from-Shading for estimating the colon depth in colonoscopy images