Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection

Yinmin Zhang,Xinzhu Ma,Shuai Yi,Jun Hou,Zhihui Wang,Wanli Ouyang,Dan Xu

2024-04-24

Abstract:As a crucial task of autonomous driving, 3D object detection has made great progress in recent years. However, monocular 3D object detection remains a challenging problem due to the unsatisfactory performance in depth estimation. Most existing monocular methods typically directly regress the scene depth while ignoring important relationships between the depth and various geometric elements (e.g. bounding box sizes, 3D object dimensions, and object poses). In this paper, we propose to learn geometry-guided depth estimation with projective modeling to advance monocular 3D object detection. Specifically, a principled geometry formula with projective modeling of 2D and 3D depth predictions in the monocular 3D object detection network is devised. We further implement and embed the proposed formula to enable geometry-aware deep representation learning, allowing effective 2D and 3D interactions for boosting the depth estimation. Moreover, we provide a strong baseline through addressing substantial misalignment between 2D annotation and projected boxes to ensure robust learning with the proposed geometric formula. Experiments on the KITTI dataset show that our method remarkably improves the detection performance of the state-of-the-art monocular-based method without extra data by 2.80% on the moderate test setting. The model and code will be released at <a class="link-external link-https" href="https://github.com/YinminZhang/MonoGeo" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily addresses the challenge of depth estimation in monocular 3D object detection by proposing an improved depth estimation method based on geometric relationships.

Specifically, the monocular 3D object detection task is highly challenging due to the lack of reliable depth information, which is crucial in applications such as autonomous driving. Although significant progress has been made in this field in recent years, monocular methods still lag behind in performance compared to methods using LiDAR or multi-view stereo vision technology, mainly due to the inaccuracy of depth information.

To improve this situation, researchers have proposed an effective method to jointly model the relationship between depth and other geometric elements, such as 2D bounding box size, 3D object dimensions, and object pose. This method leverages the principles of perspective projection to design a geometric module that can be embedded into a deep learning framework, enabling the learning of geometric relationships at the representation level. This, in turn, guides depth estimation and ultimately enhances the overall performance of monocular 3D object detection.

In summary, the paper attempts to address the issue of how to improve depth estimation in monocular 3D object detection by establishing accurate relationships between depth and other geometric elements, thereby enhancing overall detection performance.