MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors

Fanqi Pu,Yifan Wang,Jiru Deng,Wenming Yang

2024-10-25

Abstract:Perspective projection has been extensively utilized in monocular 3D object detection methods. It introduces geometric priors from 2D bounding boxes and 3D object dimensions to reduce the uncertainty of depth estimation. However, due to depth errors originating from the object's visual surface, the height of the bounding box often fails to represent the actual projected central height, which undermines the effectiveness of geometric depth. Direct prediction for the projected height unavoidably results in a loss of 2D priors, while multi-depth prediction with complex branches does not fully leverage geometric depth. This paper presents a Transformer-based monocular 3D object detection method called MonoDGP, which adopts perspective-invariant geometry errors to modify the projection formula. We also try to systematically discuss and explain the mechanisms and efficacy behind geometry errors, which serve as a simple but effective alternative to multi-depth prediction. Additionally, MonoDGP decouples the depth-guided decoder and constructs a 2D decoder only dependent on visual features, providing 2D priors and initializing object queries without the disturbance of 3D detection. To further optimize and fine-tune input tokens of the transformer decoder, we also introduce a Region Segment Head (RSH) that generates enhanced features and segment embeddings. Our monocular method demonstrates state-of-the-art performance on the KITTI benchmark without extra data. Code is available at <a class="link-external link-https" href="https://github.com/PuFanqi23/MonoDGP" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper attempts to address the issue of how to improve the accuracy of depth estimation in monocular 3D object detection. Specifically, existing monocular 3D object detection methods have some limitations when using 2D bounding boxes and 3D object dimensions to reduce depth estimation uncertainty. These limitations include:

1. **Depth Error Issue**: Due to the depth error of the object's visual surface, the height of the 2D bounding box often cannot accurately represent the actual projection center height, which weakens the effectiveness of geometric depth. Directly predicting the projection height leads to the loss of 2D prior information, and although multi-depth prediction can partially alleviate this problem, it cannot fully utilize geometric depth.

2. **Model Convergence Issue**: In the early stages of training, the accuracy of the bounding box is poor, which may cause the model to associate depth with incorrect bounding boxes, thereby affecting the model's convergence process.

To address these issues, the paper proposes a Transformer-based monocular 3D object detection method—MonoDGP (Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors). The main innovations of this method include:

- **View-Invariant Geometric Error Prediction**: By introducing geometric error to correct the projection formula, replacing multi-depth prediction, effectively alleviating the long-tail effect of depth distribution, and reducing learning complexity.
- **Decoupled Depth-Guided Decoder**: Constructing an independent 2D visual decoder to initialize queries and reference points, improving model convergence and training stability.
- **Region Segmentation Head (RSH)**: Enhancing target region features, providing segment embeddings to optimize context representation, further improving model performance.

With these improvements, MonoDGP achieves state-of-the-art performance in the KITTI benchmark test, especially excelling without additional data.