Depth Anything V2

Lihe Yang,Bingyi Kang,Zilong Huang,Zhen Zhao,Xiaogang Xu,Jiashi Feng,Hengshuang Zhao

2024-06-14

Abstract:This work presents Depth Anything V2. Without pursuing fancy techniques, we aim to reveal crucial findings to pave the way towards building a powerful monocular depth estimation model. Notably, compared with V1, this version produces much finer and more robust depth predictions through three key practices: 1) replacing all labeled real images with synthetic images, 2) scaling up the capacity of our teacher model, and 3) teaching student models via the bridge of large-scale pseudo-labeled real images. Compared with the latest models built on Stable Diffusion, our models are significantly more efficient (more than 10x faster) and more accurate. We offer models of different scales (ranging from 25M to 1.3B params) to support extensive scenarios. Benefiting from their strong generalization capability, we fine-tune them with metric depth labels to obtain our metric depth models. In addition to our models, considering the limited diversity and frequent noise in current test sets, we construct a versatile evaluation benchmark with precise annotations and diverse scenes to facilitate future research.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper mainly focuses on the improvement of monocular depth estimation models, especially the second version of Depth Anything (V2). Depth Anything V1 has some limitations such as unstable prediction for complex scenes, insufficient details, and poor handling of transparent and reflective objects. The author improves the model's performance through three key practices:

1. Using synthesized images instead of labeled real images to improve accuracy and details.
2. Increasing the capacity of the teacher model and training it only with synthesized images.
3. Utilizing a large-scale pseudo-labeled dataset of real images to teach the student model.

Compared to the latest model based on Stable Diffusion, Depth Anything V2 shows significant improvements in inference speed, parameter quantity, and depth accuracy, and provides models of different scales to adapt to various application scenarios. In addition, the author constructs a new benchmark, DA-2K, with precise annotations and diverse scenes to facilitate future research.

The paper points out that the existing test sets restrict the accurate reflection of model performance due to noise. Therefore, this new benchmark is established to address the issues of inaccurate annotation and lack of diversity in real data. Experimental results show that Depth Anything V2 significantly improves robustness, fine details, and generalization ability in complex scenes, even the smallest scale model outperforms other community models.