Abstract:This paper aims at developing a faster and more accurate solution to the amodal 3D object detection problem for indoor scenarios. The solution is achieved through a novel neural network structure which takes a pair of RGB-D images as input and delivers oriented 3D bounding boxes as the output. Such network, named 3D-SSD, has two components: hierarchical feature fusion and multi-layer prediction. The hierarchical feature fusion combines multi-scale appearance and geometric features learned from RGB-D images, which is later utilized in the multi-layer prediction for object detection. Both the accuracy and the efficiency can be improved by exploiting 2.5D representations in a synergistic way. To specifically address the shape variance of different objects, a set of 3D anchor boxes with varying physical sizes are attached to every location on the prediction layers. While testing, the category scores for 3D anchor boxes are generated with adjusted positions, sizes and orientations, leading to the final detections using non-maximum suppression. Comprehensive experiments have been performed on publicly accessible dataset of SUN RGB-D and NYUV2. The results show the proposed algorithm is the first 3D detector that runs in near real-time on the challenging datasets with competitive performance to the state-of-the-art methods. The 3D-SSD gets 37.1% mAP on the SUN RGB-D dataset at around 5.6 fps, which outperforms the state-of-the-art Deep Sliding Shape by 10.2% mAP and around 109 x faster. For an efficient model setting with a rate of 9.3 fps, 3D-SSD still gets an accuracy of 37% on mAP. Further, experiments also suggest the proposed approach achieves comparable accuracy and is about 477 x faster than the state-of-art method on the NYUv2 dataset even with a smaller input image size. (C) 2019 Published by Elsevier B.V.

Motion Detection Based on RGB-D Data and Scene Flow Clustering

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

Motion Detection for Rapidly Moving Cameras in Fully 3D Scenes

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

3D Target Detection and Tracking Based on Scene Flow

Salient-motion-heuristic Scheme for Fast 3D Optical Flow Estimation Using RGB-D Data

Moving Object Detection Based on 3D Scene Flow for Autonomous Vehicles

Layered RGBD Scene Flow Estimation with Global Non-rigid Local Rigid Assumption

Automatic Layered RGB-D Scene Flow Estimation with Optical Flow Field Constraint.

An Efficient Optical Flow Based Motion Detection Method for Non-stationary Scenes

Motion Recovery Using the Image Interpolation Algorithm and an RGB-D Camera

Handling Occlusion and Large Displacement Through Improved RGB-D Scene Flow Estimation

An effective motion object detection method using optical flow estimation under a moving camera

Abnormal Behavior Detection Based on Regional Optical Flow

Feature Regions Segmentation Based RGB-D Visual Odometry in Dynamic Environment

Moving Object Detection Based on Segmentation of Optical Flow Field in Ir Image Sequence

Detecting Motion Patterns in Dynamic Crowd Scenes.

Fast Moving Object Detection with Non-Stationary Background

Moving Object Detection Using Monocular Moving Camera with Normal Flows

Camera Motion Estimation from RGB-D-Inertial Scene Flow

Optical Flow Based Real-time Moving Object Detection in Unconstrained Scenes.