Competition for roadside camera monocular 3D object detection

Jinrang Jia,Yifeng Shi,Yuli Qu,Rui Wang,Xing Xu,Hai Zhang

DOI: https://doi.org/10.1093/nsr/nwad121

IF: 20.6

2023-05-04

National Science Review

Abstract:This perspective highlights the significance and challenges of roadside 3D monocular detection competition, and introduces the champion algorithm: a dual-stream network based on a depth map.

multidisciplinary sciences

What problem does this paper attempt to address?

Leveraging Front and Side Cues for Occlusion Handling in Monocular 3D Object Detection

Yuying Song,Zecheng Li,Jingxuan Wu,Chunyi Song,Zhiwei Xu

DOI: https://doi.org/10.1007/s00371-023-02884-0

2024-01-01

Abstract:3D object detection, as an essential part of perception, plays a principal role in the autonomous driving system. The cost-competitive monocular 3D object detection has drawn increasing attention recently. However, it still suffers an inferior accuracy especially for occluded objects due to the limited camera view. Inspired by compositional models, in which an object is represented as a combination of multiple components, this paper proposes a new monocular 3D object detection method that decreases the impact of occlusion by utilizing an object’s front and side cues. To do this, the features are extracted from a decoupled front and side representation and then fused by an attention-based module to obtain a more consistent feature distribution. An uncertainty-guided depth ensemble based on geometry is further applied to refine the depth prediction. Experiment results demonstrate that as compared to the conventional methods, the proposed method significantly improves the detection performance for occluded objects while still satisfying real-time efficiency, with the Average Precision on 40 recall positions (AP40), respectively, increasing by 10.23% for partly occluded objects and 12.22% for mostly occluded objects in the KITTI benchmark. The codes are released at https://github.com/kagurua/Front-Side-Det
MT-SSD: Single-Stage 3D Object Detector Based on Magnification Transformation

Qifeng Liu,Yabo Dong,Dawei Zhao,Liang Xiao,Bin Dai,Chen Min,Junru Zhang,Yiming Nie,Dongming Lu

DOI: https://doi.org/10.1109/TIV.2024.3400792

IF: 8.2

2024-01-01

IEEE Transactions on Intelligent Vehicles

Abstract:3D object detection plays a pivotal role in autonomous driving. Although single-stage detectors excel in speed, they often fall short in accuracy. We have identified two main issues. First, there is a significant discrepancy in prediction accuracy across different Intersection over Union (IoU) thresholds, indicating the presence of localization errors within the model. Second, traditional point-based detection models rely heavily on 1×1 convolution operations at the Set Abstraction layer, neglecting the relationship between adjacent points. To address these issues, we present the Magnification Transformation Single-Stage Detector (MT-SSD), featuring an innovative magnification Linear Transformation Module. This module applies a linear transformation to the original point cloud, sampling radius, and object labels, magnifying the error between model predictions and true values. During inference, an inverse linear transformation is applied to the detections to achieve accurate object localization. Moreover, MT-SSD introduces the Contextual Set Abstraction (CSA) layer, incorporating 1×N convolutions within the Set Abstraction layer to achieve more thorough aggregation of features among neighboring points. Our comprehensive evaluations on various autonomous driving datasets validate MT-SSD's superior performance and efficiency. Particularly noteworthy is its achievement on the Waymo Open Dataset, where MT-SSD establishes new benchmarks in single-stage 3D object detection, setting a series of state-of-the-art records. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/qifeng22/MT-SSD</uri> .
Roadside Monocular 3D Detection via 2D Detection Prompting

Yechi Ma,Shuoquan Wei,Churun Zhang,Wei Hua,Yanan Li,Shu Kong

2024-04-04

Abstract:The problem of roadside monocular 3D detection requires detecting objects of interested classes in a 2D RGB frame and predicting their 3D information such as locations in bird's-eye-view (BEV). It has broad applications in traffic control, vehicle-vehicle communication, and vehicle-infrastructure cooperative perception. To approach this problem, we present a novel and simple method by prompting the 3D detector using 2D detections. Our method builds on a key insight that, compared with 3D detectors, a 2D detector is much easier to train and performs significantly better w.r.t detections on the 2D image plane. That said, one can exploit 2D detections of a well-trained 2D detector as prompts to a 3D detector, being trained in a way of inflating such 2D detections to 3D towards 3D detection. To construct better prompts using the 2D detector, we explore three techniques: (a) concatenating both 2D and 3D detectors' features, (b) attentively fusing 2D and 3D detectors' features, and (c) encoding predicted 2D boxes x, y, width, height, label and attentively fusing such with the 3D detector's features. Surprisingly, the third performs the best. Moreover, we present a yaw tuning tactic and a class-grouping strategy that merges classes based on their functionality; these techniques improve 3D detection performance further. Comprehensive ablation studies and extensive experiments demonstrate that our method resoundingly outperforms prior works, achieving the state-of-the-art on two large-scale roadside 3D detection benchmarks.

Computer Vision and Pattern Recognition
MonoGAE: Roadside Monocular 3D Object Detection with Ground-Aware Embeddings

Lei Yang,Jiaxin Yu,Xinyu Zhang,Jun Li,Li Wang,Yi Huang,Chuang Zhang,Hong Wang,Yiming Li

2023-09-30

Abstract:Although the majority of recent autonomous driving systems concentrate on developing perception methods based on ego-vehicle sensors, there is an overlooked alternative approach that involves leveraging intelligent roadside cameras to help extend the ego-vehicle perception ability beyond the visual range. We discover that most existing monocular 3D object detectors rely on the ego-vehicle prior assumption that the optical axis of the camera is parallel to the ground. However, the roadside camera is installed on a pole with a pitched angle, which makes the existing methods not optimal for roadside scenes. In this paper, we introduce a novel framework for Roadside Monocular 3D object detection with ground-aware embeddings, named MonoGAE. Specifically, the ground plane is a stable and strong prior knowledge due to the fixed installation of cameras in roadside scenarios. In order to reduce the domain gap between the ground geometry information and high-dimensional image features, we employ a supervised training paradigm with a ground plane to predict high-dimensional ground-aware embeddings. These embeddings are subsequently integrated with image features through cross-attention mechanisms. Furthermore, to improve the detector's robustness to the divergences in cameras' installation poses, we replace the ground plane depth map with a novel pixel-level refined ground plane equation map. Our approach demonstrates a substantial performance advantage over all previous monocular 3D object detectors on widely recognized 3D detection benchmarks for roadside cameras. The code and pre-trained models will be released soon.

Computer Vision and Pattern Recognition
Three-Dimensional Detection Method of Autonomous Driving Platform Based on Roadside Monocular Camera

ZHANG Lei,WANG Weitao,WANG Bin,ZHAO Jiaqi

DOI: https://doi.org/10.3969/j.issn.1003-9767.2023.02.060

2023-01-01

Abstract:Aiming at the self-positioning problem of autonomous driving platform, combined with Vehicle-road collaboration technology, a three-dimensional detection method based on roadside monocular camera is proposed. Firstly, the parameters of the monocular camera are calibrated. Secondly, YOLOv7 model is used for two-dimensional target detection.Finally, the module based on calibration is used to obtain the three-dimensional coordinate of the autonomous driving platform.The above method fully considers the limitation of edge computing resources and the requirement of detection speed, and can quickly obtain the auxiliary positioning information of autonomous driving platform. It provides the decision basis for the platform fusion positioning, which is helpful to further improve the positioning accuracy of platform in the road section with weak signal.
Monocular Visual Object 3D Localization in Road Scenes

Yizhou Wang,Yen-Ting Huang,Jenq-Neng Hwang

DOI: https://doi.org/10.1145/3343031.3350924

2019-01-01

Abstract:3D localization of objects in road scenes is important for autonomous driving and advanced driver-assistance systems (ADAS). However, with common monocular camera setups, 3D information is difficult to obtain. In this paper, we propose a novel and robust method for 3D localization of monocular visual objects in road scenes by joint integration of depth estimation, ground plane estimation, and multi-object tracking techniques. Firstly, an object depth estimation method with depth confidence is proposed by utilizing the monocular depthmap from a CNN. Secondly, an adaptive ground plane estimation using both dense and sparse features is proposed to localize the objects when their depth estimation is not reliable. Thirdly, temporal information is taken into consideration by a new object tracklet smoothing method. Unlike most existing methods which only consider vehicle localization, our method is applicable for common moving objects in the road scenes, including pedestrians, vehicles, cyclists, etc. Moreover, the input depthmap can be replaced by some equivalent depth information from other sensors, like LiDAR, depth camera and Radar, which makes our system much more competitive compared with other object localization methods. As evaluated on KITTI dataset, our method achieves favorable performance on 3D localization of both pedestrians and vehicles when compared with the state-of-the-art vehicle localization methods, though no published performance on pedestrian 3D localization can be compared with, from the best of our knowledge.
Ground-aware Monocular 3D Object Detection for Autonomous Driving

Yuxuan Liu,Yuan Yixuan,Ming Liu

DOI: https://doi.org/10.48550/arXiv.2102.00690

2021-02-01

Abstract:Estimating the 3D position and orientation of objects in the environment with a single RGB camera is a critical and challenging task for low-cost urban autonomous driving and mobile robots. Most of the existing algorithms are based on the geometric constraints in 2D-3D correspondence, which stems from generic 6D object pose estimation. We first identify how the ground plane provides additional clues in depth reasoning in 3D detection in driving scenes. Based on this observation, we then improve the processing of 3D anchors and introduce a novel neural network module to fully utilize such application-specific priors in the framework of deep learning. Finally, we introduce an efficient neural network embedded with the proposed module for 3D object detection. We further verify the power of the proposed module with a neural network designed for monocular depth prediction. The two proposed networks achieve state-of-the-art performances on the KITTI 3D object detection and depth prediction benchmarks, respectively. The code will be published in <a class="link-external link-https" href="https://www.github.com/Owen-Liuyuxuan/visualDet3D" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition,Robotics
CenterLoc3D: Monocular 3D Vehicle Localization Network for Roadside Surveillance Cameras

Tang Xinyao,Wang Wei,Song Huansheng,Zhao Chunhui

DOI: https://doi.org/10.48550/arXiv.2203.14550

2023-01-05

Abstract:Monocular 3D vehicle localization is an important task in Intelligent Transportation System (ITS) and Cooperative Vehicle Infrastructure System (CVIS), which is usually achieved by monocular 3D vehicle detection. However, depth information cannot be obtained directly by monocular cameras due to the inherent imaging mechanism, resulting in more challenging monocular 3D tasks. Most of the current monocular 3D vehicle detection methods leverage 2D detectors and additional geometric modules, which reduces the efficiency. In this paper, we propose a 3D vehicle localization network CenterLoc3D for roadside monocular cameras, which directly predicts centroid and eight vertexes in image space, and the dimension of 3D bounding boxes without 2D detectors. To improve the precision of 3D vehicle localization, we propose a weighted-fusion module and a loss with spatial constraints embedded in CenterLoc3D. Firstly, the transformation matrix between 2D image space and 3D world space is solved by camera calibration. Secondly, vehicle type, centroid, eight vertexes, and the dimension of 3D vehicle bounding boxes are obtained by CenterLoc3D. Finally, centroid in 3D world space can be obtained by camera calibration and CenterLoc3D for 3D vehicle localization. To the best of our knowledge, this is the first application of 3D vehicle localization for roadside monocular cameras. Hence, we also propose a benchmark for this application including a dataset (SVLD-3D), an annotation tool (LabelImg-3D), and evaluation metrics. Through experimental validation, the proposed method achieves high accuracy and real-time performance. (limited words, please see the article for more details)

Computer Vision and Pattern Recognition,Artificial Intelligence
HeightFormer: A Semantic Alignment Monocular 3D Object Detection Method from Roadside Perspective

Pei Liu,Zihao Zhang,Haipeng Liu,Nanfang Zheng,Meixin Zhu,Ziyuan Pu

2024-10-21

Abstract:The on-board 3D object detection technology has received extensive attention as a critical technology for autonomous driving, while few studies have focused on applying roadside sensors in 3D traffic object detection. Existing studies achieve the projection of 2D image features to 3D features through height estimation based on the frustum. However, they did not consider the height alignment and the extraction efficiency of bird's-eye-view features. We propose a novel 3D object detection framework integrating Spatial Former and Voxel Pooling Former to enhance 2D-to-3D projection based on height estimation. Extensive experiments were conducted using the Rope3D and DAIR-V2X-I dataset, and the results demonstrated the outperformance of the proposed algorithm in the detection of both vehicles and cyclists. These results indicate that the algorithm is robust and generalized under various detection scenarios. Improving the accuracy of 3D object detection on the roadside is conducive to building a safe and trustworthy intelligent transportation system of vehicle-road coordination and promoting the large-scale application of autonomous driving. The code and pre-trained models will be released on <a class="link-external link-https" href="https://anonymous.4open.science/r/HeightFormer" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
Pseudo-Mono for Monocular 3D Object Detection in Autonomous Driving

Chongben Tao,JieCheng Cao,Chen Wang,Zufeng Zhang,Zhen Gao

DOI: https://doi.org/10.1109/tcsvt.2023.3237579

IF: 5.859

2023-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Current monocular 3D object detection algorithms generally suffer from inaccurate depth estimation, which leads to reduction of detection accuracy. The depth error from image-to-image generation for the stereo view is insignificant compared with the gap in single-image generation. Therefore, a novel pseudo-monocular 3D object detection framework is proposed, which is called Pseudo-Mono. Particularly, stereo images are brought into monocular 3D detection. Firstly, stereo images are taken as input, then a lightweight depth predictor is used to generate the depth map of input images. Secondly, the left input images obtained from stereo camera are used as subjects, which generate enhanced visual feature and multi-scale depth feature by depth indexing and feature matching probabilities, respectively. Finally, sparse anchors set by the foreground probability maps and the multi-scale feature maps are used as reference points to find the suitable initialization approach of object query. The encoded visual feature is adopted to enhance object query for enabling deep interaction between visual feature and depth feature. Compared with popular monocular 3D object detection methods, Pseudo-Mono is able to achieve richer fine-grained information without additional data input. Extensive experimental results on the datasets of KITTI, NuScenes, and MS-COCO demonstrate the generalizability and portability of the proposed method. The effectiveness and efficiency of Pseudo-Mono have been demonstrated by extensive ablation experiments. Experiments on a real vehicle platform have shown that the proposed method maintains high performance in complex real-world environments.

engineering, electrical & electronic
Monocular 3D Detection for Autonomous Vehicles by Cascaded Geometric Constraints and Depurated Using 3D Results

Fang Jiaojiao,Zhou Linglao,Liu Guizhong

DOI: https://doi.org/10.1109/icus50048.2020.9274863

2020-01-01

Abstract:3D object detection is a key task in 3D vision perception of autonomous vehicles. In this paper, we present a novel two-stage 3D object detection method aimed to get a more accurate 3D location of an object. We modify existing 3D properties regressing network by adding two additional components, viewpoints classification and the center projection regression of a 3D box's bottom face (CBF). The center projection is associated with a similar triangle constraint to acquire an initial 3D location of a closed-form solution. For no truncated objects, the previous predicted location is involved in the initial value of over-determined equations constructed by the 2D-3D boxes fitting constraint with the configuration determined by the classified viewpoint. Then the recovered 3D information is utilized to purify the detection results. Results of comparison with state-of-the-art methods on the KITTI dataset show that although conceptually simple, our method outperforms more complex and computationally expensive methods. Furthermore, our method can filter out false alarms and false detection in both 2D and 3D results.
Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Driving

Yi-Nan Chen,Hang Dai,Yong Ding

DOI: https://doi.org/10.48550/arXiv.2203.02112

2022-03-04

Abstract:Pseudo-LiDAR 3D detectors have made remarkable progress in monocular 3D detection by enhancing the capability of perceiving depth with depth estimation networks, and using LiDAR-based 3D detection architectures. The advanced stereo 3D detectors can also accurately localize 3D objects. The gap in image-to-image generation for stereo views is much smaller than that in image-to-LiDAR generation. Motivated by this, we propose a Pseudo-Stereo 3D detection framework with three novel virtual view generation methods, including image-level generation, feature-level generation, and feature-clone, for detecting 3D objects from a single image. Our analysis of depth-aware learning shows that the depth loss is effective in only feature-level virtual view generation and the estimated depth map is effective in both image-level and feature-level in our framework. We propose a disparity-wise dynamic convolution with dynamic kernels sampled from the disparity feature map to filter the features adaptively from a single image for generating virtual image features, which eases the feature degradation caused by the depth estimation errors. Till submission (November 18, 2021), our Pseudo-Stereo 3D detection framework ranks 1st on car, pedestrian, and cyclist among the monocular 3D detectors with publications on the KITTI-3D benchmark. The code is released at <a class="link-external link-https" href="https://github.com/revisitq/Pseudo-Stereo-3D" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving

Xinzhu Ma,Zhihui Wang,Haojie Li,Pengbo Zhang,Xin Fan,Wanli Ouyang

DOI: https://doi.org/10.48550/arXiv.1903.11444

2021-03-30

Abstract:In this paper, we propose a monocular 3D object detection framework in the domain of autonomous driving. Unlike previous image-based methods which focus on RGB feature extracted from 2D images, our method solves this problem in the reconstructed 3D space in order to exploit 3D contexts explicitly. To this end, we first leverage a stand-alone module to transform the input data from 2D image plane to 3D point clouds space for a better input representation, then we perform the 3D detection using PointNet backbone net to obtain objects 3D locations, dimensions and orientations. To enhance the discriminative capability of point clouds, we propose a multi-modal feature fusion module to embed the complementary RGB cue into the generated point clouds representation. We argue that it is more effective to infer the 3D bounding boxes from the generated 3D scene space (i.e., X,Y, Z space) compared to the image plane (i.e., R,G,B image plane). Evaluation on the challenging KITTI dataset shows that our approach boosts the performance of state-of-the-art monocular approach by a large margin.

Computer Vision and Pattern Recognition
SGM3D: Stereo Guided Monocular 3D Object Detection

Zheyuan Zhou,Liang Du,Xiaoqing Ye,Zhikang Zou,Xiao Tan,Li Zhang,Xiangyang Xue,Jianfeng Feng

DOI: https://doi.org/10.1109/LRA.2022.3191849

2022-02-25

Abstract:Monocular 3D object detection aims to predict the object location, dimension and orientation in 3D space alongside the object category given only a monocular image. It poses a great challenge due to its ill-posed property which is critically lack of depth information in the 2D image plane. While there exist approaches leveraging off-the-shelve depth estimation or relying on LiDAR sensors to mitigate this problem, the dependence on the additional depth model or expensive equipment severely limits their scalability to generic 3D perception. In this paper, we propose a stereo-guided monocular 3D object detection framework, dubbed SGM3D, adapting the robust 3D features learned from stereo inputs to enhance the feature for monocular detection. We innovatively present a multi-granularity domain adaptation (MG-DA) mechanism to exploit the network's ability to generate stereo-mimicking features given only on monocular cues. Coarse BEV feature-level, as well as the fine anchor-level domain adaptation, are both leveraged for guidance in the monocular <a class="link-external link-http" href="http://domain.In" rel="external noopener nofollow">this http URL</a> addition, we introduce an IoU matching-based alignment (IoU-MA) method for object-level domain adaptation between the stereo and monocular predictions to alleviate the mismatches while adopting the MG-DA. Extensive experiments demonstrate state-of-the-art results on KITTI and Lyft datasets.

Computer Vision and Pattern Recognition
Neighbor-Vote: Improving Monocular 3D Object Detection through Neighbor Distance Voting

Xiaomeng Chu,Jiajun Deng,Yao Li,Zhenxun Yuan,Yanyong Zhang,Jianmin Ji,Yu Zhang

DOI: https://doi.org/10.48550/arXiv.2107.02493

2021-07-06

Abstract:As cameras are increasingly deployed in new application domains such as autonomous driving, performing 3D object detection on monocular images becomes an important task for visual scene understanding. Recent advances on monocular 3D object detection mainly rely on the ``pseudo-LiDAR'' generation, which performs monocular depth estimation and lifts the 2D pixels to pseudo 3D points. However, depth estimation from monocular images, due to its poor accuracy, leads to inevitable position shift of pseudo-LiDAR points within the object. Therefore, the predicted bounding boxes may suffer from inaccurate location and deformed shape. In this paper, we present a novel neighbor-voting method that incorporates neighbor predictions to ameliorate object detection from severely deformed pseudo-LiDAR point clouds. Specifically, each feature point around the object forms their own predictions, and then the ``consensus'' is achieved through voting. In this way, we can effectively combine the neighbors' predictions with local prediction and achieve more accurate 3D detection. To further enlarge the difference between the foreground region of interest (ROI) pseudo-LiDAR points and the background points, we also encode the ROI prediction scores of 2D foreground pixels into the corresponding pseudo-LiDAR points. We conduct extensive experiments on the KITTI benchmark to validate the merits of our proposed method. Our results on the bird's eye view detection outperform the state-of-the-art performance by a large margin, especially for the ``hard'' level detection.

Computer Vision and Pattern Recognition
Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud

Xinshuo Weng,Kris Kitani

DOI: https://doi.org/10.48550/arXiv.1903.09847

2019-08-31

Abstract:Monocular 3D scene understanding tasks, such as object size estimation, heading angle estimation and 3D localization, is challenging. Successful modern day methods for 3D scene understanding require the use of a 3D sensor. On the other hand, single image based methods have significantly worse performance. In this work, we aim at bridging the performance gap between 3D sensing and 2D sensing for 3D object detection by enhancing LiDAR-based algorithms to work with single image input. Specifically, we perform monocular depth estimation and lift the input image to a point cloud representation, which we call pseudo-LiDAR point cloud. Then we can train a LiDAR-based 3D detection network with our pseudo-LiDAR end-to-end. Following the pipeline of two-stage 3D detection algorithms, we detect 2D object proposals in the input image and extract a point cloud frustum from the pseudo-LiDAR for each proposal. Then an oriented 3D bounding box is detected for each frustum. To handle the large amount of noise in the pseudo-LiDAR, we propose two innovations: (1) use a 2D-3D bounding box consistency constraint, adjusting the predicted 3D bounding box to have a high overlap with its corresponding 2D proposal after projecting onto the image; (2) use the instance mask instead of the bounding box as the representation of 2D proposals, in order to reduce the number of points not belonging to the object in the point cloud frustum. Through our evaluation on the KITTI benchmark, we achieve the top-ranked performance on both bird's eye view and 3D object detection among all monocular methods, effectively quadrupling the performance over previous state-of-the-art. Our code is available at <a class="link-external link-https" href="https://github.com/xinshuoweng/Mono3D_PLiDAR" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
Monocular 3D Object Detection With Sequential Feature Association and Depth Hint Augmentation

Tianze Gao,Huihui Pan,Huijun Gao

DOI: https://doi.org/10.1109/TIV.2022.3143954

IF: 8.2

2022-01-01

IEEE Transactions on Intelligent Vehicles

Abstract:Monocular 3D object detection, with the aim of predicting the geometric properties of on-road objects, is a promising research topic for the intelligent perception systems of autonomous driving. Most state-of-the-art methods follow a keypoint-based paradigm, where the keypoints of objects are predicted and employed as the basis for regressing the other geometric properties. In this work, a unified network named as FADNet is presented to address the task of monocular 3D object detection. In contrast to previous keypoint-based methods, we propose to divide the output modalities into different groups according to the estimation difficulty of object properties. Different groups are treated differently and sequentially associated by a convolutional Gated Recurrent Unit. Another contribution of this work is the strategy of depth hint augmentation. To provide characterized depth patterns as hints for depth estimation, a dedicated depth hint module is designed to generate row-wise features named as depth hints, which are explicitly supervised in a bin-wise manner. The contributions of this work are validated by conducting experiments and ablation study on the KITTI benchmark. Without utilizing depth priors, post optimization, or other refinement modules, our network performs competitively against state-of-the-art methods while maintaining a decent running speed.
Monocular 3-D Vehicle Detection Using a Cascade Network for Autonomous Driving

Hongyu Hu,Tongtong Zhao,Qi Wang,Fei Gao,Lei He,Zhenhai Gao

DOI: https://doi.org/10.1109/tim.2021.3094622

IF: 5.6

2021-01-01

IEEE Transactions on Instrumentation and Measurement

Abstract:Three-dimensional object detection plays an important role in autonomous driving because it provides the 3-D locations of objects for subsequent use in decision-making modules. A novel method is proposed using a monocular image and cascade geometric constraints to achieve robust 3-D vehicle detection. The framework is divided into two stages. In the first stage, the monocular image input is processed using a heatmap-based detection network with five branches to regress the orientation, dimension, center projection of the bottom face, viewpoint classification, and 2-D bounding box. In the second stage, the intersection-over-union threshold is increased to filter out imprecise 2-D bounding boxes. Thereafter, cascade geometric constraints are used to obtain the final 3-D box output, which improves the detection performance under truncation and occlusion conditions. The proposed method is tested on the KITTI-3-D benchmark and is shown to be effective and efficient. The proposed framework does not depend on external sources or subnetworks and can be trained in an end-to-end manner.

engineering, electrical & electronic,instruments & instrumentation
Monocular 3D Object Detection: An Extrinsic Parameter Free Approach

Yunsong Zhou,Yuan He,Hongzi Zhu,Cheng Wang,Hongyang Li,Qinhong Jiang

DOI: https://doi.org/10.48550/arXiv.2106.15796

2021-06-30

Computer Vision and Pattern Recognition

Abstract:Monocular 3D object detection is an important task in autonomous driving. It can be easily intractable where there exists ego-car pose change w.r.t. ground plane. This is common due to the slight fluctuation of road smoothness and slope. Due to the lack of insight in industrial application, existing methods on open datasets neglect the camera pose information, which inevitably results in the detector being susceptible to camera extrinsic parameters. The perturbation of objects is very popular in most autonomous driving cases for industrial products. To this end, we propose a novel method to capture camera pose to formulate the detector free from extrinsic perturbation. Specifically, the proposed framework predicts camera extrinsic parameters by detecting vanishing point and horizon change. A converter is designed to rectify perturbative features in the latent space. By doing so, our 3D detector works independent of the extrinsic parameter variations and produces accurate results in realistic cases, e.g., potholed and uneven roads, where almost all existing monocular detectors fail to handle. Experiments demonstrate our method yields the best performance compared with the other state-of-the-arts by a large margin on both KITTI 3D and nuScenes datasets.
RTM3D: Real-Time Monocular 3D Detection from Object Keypoints for Autonomous Driving

Peixuan Li,Huaici Zhao,Pengfei Liu,Feidao Cao

DOI: https://doi.org/10.1007/978-3-030-58580-8_38

2020-01-01

Abstract:In this work, we propose an efficient and accurate monocular 3D detection framework in single shot. Most successful 3D detectors take the projection constraint from the 3D bounding box to the 2D box as an important component. Four edges of a 2D box provide only four constraints and the performance deteriorates dramatically with the small error of the 2D detector. Different from these approaches, our method predicts the nine perspective keypoints of a 3D bounding box in image space, and then utilize the geometric relationship of 3D and 2D perspectives to recover the dimension, location, and orientation in 3D space. In this method, the properties of the object can be predicted stably even when the estimation of keypoints is very noisy, which enables us to obtain fast detection speed with a small architecture. Training our method only uses the 3D properties of the object without any extra annotations, category-specific 3D shape priors, or depth maps. Our method is the first real-time system (FPS > 24) for monocular image 3D detection while achieves state-of-the-art performance on the KITTI benchmark.

Competition for roadside camera monocular 3D object detection

Leveraging Front and Side Cues for Occlusion Handling in Monocular 3D Object Detection

MT-SSD: Single-Stage 3D Object Detector Based on Magnification Transformation

Roadside Monocular 3D Detection via 2D Detection Prompting

MonoGAE: Roadside Monocular 3D Object Detection with Ground-Aware Embeddings

Three-Dimensional Detection Method of Autonomous Driving Platform Based on Roadside Monocular Camera

Monocular Visual Object 3D Localization in Road Scenes

Ground-aware Monocular 3D Object Detection for Autonomous Driving

CenterLoc3D: Monocular 3D Vehicle Localization Network for Roadside Surveillance Cameras

HeightFormer: A Semantic Alignment Monocular 3D Object Detection Method from Roadside Perspective

Pseudo-Mono for Monocular 3D Object Detection in Autonomous Driving

Monocular 3D Detection for Autonomous Vehicles by Cascaded Geometric Constraints and Depurated Using 3D Results

Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Driving

Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving

SGM3D: Stereo Guided Monocular 3D Object Detection

Neighbor-Vote: Improving Monocular 3D Object Detection through Neighbor Distance Voting

Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud

Monocular 3D Object Detection With Sequential Feature Association and Depth Hint Augmentation

Monocular 3-D Vehicle Detection Using a Cascade Network for Autonomous Driving

Monocular 3D Object Detection: An Extrinsic Parameter Free Approach

RTM3D: Real-Time Monocular 3D Detection from Object Keypoints for Autonomous Driving