Abstract:Monocular 3D object detection (Mono3D) in mobile settings (e.g., on a vehicle, a drone, or a robot) is an important yet challenging task. Due to the near-far disparity phenomenon of monocular vision and the ever-changing camera pose, it is hard to acquire high detection accuracy, especially for far objects. Inspired by the insight that the depth of an object can be well determined according to the depth of the ground where it stands, in this paper, we propose a novel Mono3D framework, called MoGDE, which constantly estimates the corresponding ground depth of an image and then utilizes the estimated ground depth information to guide Mono3D. To this end, we utilize a pose detection network to estimate the pose of the camera and then construct a feature map portraying pixel-level ground depth according to the 3D-to-2D perspective geometry. Moreover, to improve Mono3D with the estimated ground depth, we design an RGB-D feature fusion network based on the transformer structure, where the long-range self-attention mechanism is utilized to effectively identify ground-contacting points and pin the corresponding ground depth to the image feature map. We conduct extensive experiments on the real-world KITTI dataset. The results demonstrate that MoGDE can effectively improve the Mono3D accuracy and robustness for both near and far objects. MoGDE yields the best performance compared with the state-of-the-art methods by a large margin and is ranked number one on the KITTI 3D benchmark.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the problem of monocular 3D object detection (Mono3D) in mobile environments. Specifically, the authors focus on the issue of low detection accuracy, especially for distant objects, caused by the near-far disparity phenomenon of monocular vision and the constantly changing camera pose when performing monocular 3D object detection on mobile devices such as vehicles, drones, or robots. ### Main Contributions 1. **Proposed a new monocular 3D detection framework MoGDE**: By dynamically estimating ground depth information as prior knowledge, it improves the detection accuracy and robustness for both near and distant objects. 2. **Designed a Transformer-based feature fusion network**: Utilizing long-range self-attention mechanisms to effectively identify the contact points between objects and the ground, and attach the corresponding ground depth information to the image feature map. 3. **Conducted extensive experiments on the real-world KITTI dataset**: The results show that MoGDE significantly outperforms existing state-of-the-art methods on various metrics and ranks first in the KITTI 3D benchmark. ### Method Overview 1. **Ground Depth Estimation (GDE)**: - Predicts the camera pose (e.g., pitch and roll angles) through a pose detection network and extracts vanishing point and ground plane information. - Constructs a virtual 3D scene and generates a pixel-level ground depth feature map based on the 3D-to-2D perspective geometry relationship. 2. **Ground Depth Fusion (GDF)**: - Uses the attention mechanism of the Transformer network to fuse image features with the ground depth feature map, generating a fused feature map containing ground information. 3. **Monocular 3D Detection (M3D)**: - Uses GUPNet as the base detector and performs 3D object detection by combining the fused feature map. ### Experimental Results - **Quantitative Results**: In the KITTI 3D object detection benchmark, MoGDE achieved the highest scores across all difficulty levels, particularly improving by 17% over the second-best model in the "hard" category. - **Qualitative Results**: Visualization results on the KITTI Odometry dataset show that MoGDE's predictions are closer to the ground truth, especially in locating distant objects. ### Ablation Studies - **Effectiveness Analysis**: Ablation studies validated the effectiveness of each component, including considering camera pose changes, using ground plane cues, and adopting the Transformer-based feature fusion method. ### Conclusion By introducing dynamic ground depth information and a Transformer-based feature fusion method, this paper significantly improves the accuracy and robustness of monocular 3D object detection in mobile environments. These improvements have important value in practical applications, particularly in fields such as autonomous driving, drones, and robotics.

MoGDE: Boosting Mobile Monocular 3D Object Detection with Ground Depth Estimation

MonoDETR: Depth-guided Transformer for Monocular 3D Object Detection

SGM3D: Stereo Guided Monocular 3D Object Detection

MonoGround: Detecting Monocular 3D Objects from the Ground

MonoDTR: Monocular 3D Object Detection with Depth-Aware Transformer

MonoPGC: Monocular 3D Object Detection with Pixel Geometry Contexts

MonoSAID: Monocular 3D Object Detection Based on Scene-Level Adaptive Instance Depth Estimation

Depth-Enhancement Network for Monocular 3D object detection

MDS-Net: Multi-Scale Depth Stratification 3D Object Detection from Monocular Images

Ground-aware Monocular 3D Object Detection for Autonomous Driving

An Algorithm on Monocular 3D Object Detection Based on Depth Estimation

MonoCD: Monocular 3D Object Detection with Complementary Depths

MonoBooster: Semi-Dense Skip Connection with Cross-Level Attention for Boosting Self-Supervised Monocular Depth Estimation

MonoGRNet: A General Framework for Monocular 3D Object Detection

MonoMM: A Multi-scale Mamba-Enhanced Network for Real-time Monocular 3D Object Detection

MonoGAE: Roadside Monocular 3D Object Detection with Ground-Aware Embeddings

Pseudo-Mono for Monocular 3D Object Detection in Autonomous Driving

Depth-Vision-Decoupled Transformer With Cascaded Group Convolutional Attention for Monocular 3-D Object Detection

Mono3DVG: 3D Visual Grounding in Monocular Images

ODM3D: Alleviating Foreground Sparsity for Semi-Supervised Monocular 3D Object Detection

MonoAux: Fully Exploiting Auxiliary Information and Uncertainty for Monocular 3D Object Detection