Abstract:This paper aims at developing a faster and more accurate solution to the amodal 3D object detection problem for indoor scenarios. The solution is achieved through a novel neural network structure which takes a pair of RGB-D images as input and delivers oriented 3D bounding boxes as the output. Such network, named 3D-SSD, has two components: hierarchical feature fusion and multi-layer prediction. The hierarchical feature fusion combines multi-scale appearance and geometric features learned from RGB-D images, which is later utilized in the multi-layer prediction for object detection. Both the accuracy and the efficiency can be improved by exploiting 2.5D representations in a synergistic way. To specifically address the shape variance of different objects, a set of 3D anchor boxes with varying physical sizes are attached to every location on the prediction layers. While testing, the category scores for 3D anchor boxes are generated with adjusted positions, sizes and orientations, leading to the final detections using non-maximum suppression. Comprehensive experiments have been performed on publicly accessible dataset of SUN RGB-D and NYUV2. The results show the proposed algorithm is the first 3D detector that runs in near real-time on the challenging datasets with competitive performance to the state-of-the-art methods. The 3D-SSD gets 37.1% mAP on the SUN RGB-D dataset at around 5.6 fps, which outperforms the state-of-the-art Deep Sliding Shape by 10.2% mAP and around 109 x faster. For an efficient model setting with a rate of 9.3 fps, 3D-SSD still gets an accuracy of 37% on mAP. Further, experiments also suggest the proposed approach achieves comparable accuracy and is about 477 x faster than the state-of-art method on the NYUv2 dataset even with a smaller input image size. (C) 2019 Published by Elsevier B.V.

Fast Human Detection in RGB-D Images Based on Color-Depth Joint Feature Learning.

An ultra-fast human detection method for color-depth camera

A Two-Stage Human Body Detector on Depth Data

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

A new depth descriptor for pedestrian detection in RGB-D images

Reliably Detecting Humans in Crowded and Dynamic Environments Using RGB-D Camera

Human face detection under weak light based on depth information

Physical Blob Detector and Multi-Channel Color Shape Descriptor for Human Detection.

Detecting Humans in RGB-D Data with CNNs

Robust Human Detection with Super-Pixel Segmentation and Random Ferns Classification Using RGB-D Camera

Real-time Detection and Tracking Algorithm Based on the Color and Character of Human Face

Fast and Efficient Integration of Human Upper-Body Detection and Orientation Estimation in RGB-D Video

Detecting and tracking people in real time with RGB-D camera

Asymmetric Adaptive Fusion in a Two-Stream Network for RGB-D Human Detection

Real-time human detection and tracking in complex environments using single RGBD camera

Robust Real-Time Human Perception with Depth Camera.

Dynamic human object recognition by combining color and depth information with a clothing image histogram

Robust 3-D Human Detection in Complex Environments With a Depth Camera.

An RGB-D Based Approach for Human Pose Estimation

Reliably Detecting Humans With Rgb-D Camera With Physical Blob Detector Followed By Learning-Based Filtering

Upper Body Human Detection and Segmentation in Low Contrast Video