Abstract:This paper aims at developing a faster and more accurate solution to the amodal 3D object detection problem for indoor scenarios. The solution is achieved through a novel neural network structure which takes a pair of RGB-D images as input and delivers oriented 3D bounding boxes as the output. Such network, named 3D-SSD, has two components: hierarchical feature fusion and multi-layer prediction. The hierarchical feature fusion combines multi-scale appearance and geometric features learned from RGB-D images, which is later utilized in the multi-layer prediction for object detection. Both the accuracy and the efficiency can be improved by exploiting 2.5D representations in a synergistic way. To specifically address the shape variance of different objects, a set of 3D anchor boxes with varying physical sizes are attached to every location on the prediction layers. While testing, the category scores for 3D anchor boxes are generated with adjusted positions, sizes and orientations, leading to the final detections using non-maximum suppression. Comprehensive experiments have been performed on publicly accessible dataset of SUN RGB-D and NYUV2. The results show the proposed algorithm is the first 3D detector that runs in near real-time on the challenging datasets with competitive performance to the state-of-the-art methods. The 3D-SSD gets 37.1% mAP on the SUN RGB-D dataset at around 5.6 fps, which outperforms the state-of-the-art Deep Sliding Shape by 10.2% mAP and around 109 x faster. For an efficient model setting with a rate of 9.3 fps, 3D-SSD still gets an accuracy of 37% on mAP. Further, experiments also suggest the proposed approach achieves comparable accuracy and is about 477 x faster than the state-of-art method on the NYUv2 dataset even with a smaller input image size. (C) 2019 Published by Elsevier B.V.

Deep learning of directional truncated signed distance function for robust 3D object recognition

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

An End-to-End Deep Learning Network for 3D Object Detection From RGB-D Data Based on Hough Voting

Exploiting Label Uncertainty for Enhanced 3D Object Detection From Point Clouds

3D Street Object Detection from Monocular Images Using Deep Learning and Depth Information

Deep Hough Voting for 3D Object Detection in Point Clouds

6DoF-3D: Efficient and accurate 3D object detection using six degrees-of-freedom for autonomous driving

DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization

Point-DETR3D: Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection

Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection

Recognizing multi-view objects with occlusions using a deep architecture

SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images

S-VoteNet: Deep Hough Voting with Spherical Proposal for 3D Object Detection.

ODSPC: deep learning-based 3D object detection using semantic point cloud

Neighbor-Vote: Improving Monocular 3D Object Detection through Neighbor Distance Voting

Feature Decoupling and Uncertainty Estimation for 3D Object Detection

DSGN: Deep Stereo Geometry Network for 3D Object Detection

Shape Prior Guided Instance Disparity Estimation for 3D Object Detection

Densely Constrained Depth Estimator for Monocular 3D Object Detection

DCDet: Dynamic Cross-based 3D Object Detector

Objects are Different: Flexible Monocular 3D Object Detection