Abstract:Background and objective: Depth estimation has very broad potential in medical image analysis and is important for applications such as augmented reality surgical navigation and preoperative planning. Compared with segmentation tasks that can obtain ground truth through manual annotation, it is difficult to obtain a large number of real values for depth estimation tasks that are limited by hardware conditions in endoscopic environments. Methods: To address the challenge, we propose a novel framework that utilizes segmentation tasks to improve encoder performance in a self-supervised depth estimation network. For the first time, we leverage the Contrastive Language-Image Pre-training (CLIP) method to improve the performance of endoscopy segmentation models. Depth estimation networks can also benefit from this training process indirectly. In addition, we design a semantic-guidance loss function to improve the performance. Results: Our proposed method is systematically evaluated on three datasets. Experiments have verified that the proposed framework can assist the network model in obtaining smaller errors. Compared with other state-of-the-art methods, our framework obtains 0.081 and 0.097 on absolute relative error metrics in quantitative evaluations on SCARED and SERV-CT datasets respectively. In qualitative experiments on real surgery datasets, our proposed method also shows more ideal results. Conclusion: The experiments in this study illustrate that our proposed method can alleviate the problem of difficulty in improving network performance due to the lack of real values of depth data. The visual performance of our approach illustrates the application potential in the clinic. Our method helps doctors obtain depth perception and visual cues simultaneously, thereby reducing the difficulty of surgery and the pain of patients.

Multi-scale, Multi-Dimensional Binocular Endoscopic Image Depth Estimation Network.

A Three-Dimensional Measurement Method for Binocular Endoscopes Based on Deep Learning

Binocular Depth Estimation Using Convolutional Neural Network With Siamese Branches.

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Monocular Depth Estimation Based on Multi-Scale Graph Convolution Networks

Self-Supervised Monocular Depth Estimation for Endoscopic Imaging

Distilled Visual and Robot Kinematics Embeddings for Metric Depth Estimation in Monocular Scene Reconstruction

Advancing Depth Anything Model for Unsupervised Monocular Depth Estimation in Endoscopy

Image Intrinsic-Based Unsupervised Monocular Depth Estimation in Endoscopy

Self-Supervised Micro-Baseline Binocular Endoscope Depth Estimation Method with Domain Adaptation

Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos

Enhanced Scale-aware Depth Estimation for Monocular Endoscopic Scenes with Geometric Modeling

EndoPerfect: A Hybrid NeRF-Stereo Vision Approach Pioneering Monocular Depth Estimation and 3D Reconstruction in Endoscopy

Depth estimation from monocular endoscopy using simulation and image transfer approach

Self-supervised endoscopy depth estimation framework with CLIP-guidance segmentation

Joint estimation of depth and motion from a monocular endoscopy image sequence using a multi-loss rebalancing network

Depth-Guided Aggregation for Real-Time Binocular Depth Estimation Network

A geometry-aware deep network for depth estimation in monocular endoscopy

EndoDepthL: Lightweight Endoscopic Monocular Depth Estimation with CNN-Transformer

PADENet: an Efficient and Robust Panoramic Monocular Depth Estimation Network for Outdoor Scenes.