Polyp Size Estimation by Generalizing Metric Depth Estimation and Monocular 3D Reconstruction

Sijia Du,Qingwei Zhang,Zhengjie Zhang,Crystal Cai,XiaoBo Li,Dahong Qian
DOI: https://doi.org/10.1109/isbi56570.2024.10635184
2024-01-01
Abstract:Accurate polyp size estimation is an important step in colorectal cancer screening. However, manual estimation from monocular endoscopic images still remains a challenge for endoscopists. Vision-based methods are promising ways to estimate polyp size without interrupting examination, which calculate size from 3D information provided by depth map. However, two main challenges arise. Firstly, most metric depth networks show poor generalization to unseen clinical polyps. Additionally, the angle at which polyps are viewed may be unknown and as a consequence, the maximum polyp diameter may not be reflected on the image plane. To deal with the problems mentioned above, we propose a novel metric depth estimation framework to boost the generalization ability of metric depth estimation and a 3D polyp size estimation method to estimate the size (in cm) of the polyp from endoscopy images in a camera-angle-independent way. Based on that, our pipeline can estimate the polyp size from a single or very few images of polyps taken during clinical colonoscopies with a mean absolute error (MAE) of 0.244cm, which can strongly support clinical diagnostics. Code will be available at https://github.com/Scarlett213/PolypSizeEstimation.
What problem does this paper attempt to address?