Abstract:This paper aims at developing a faster and more accurate solution to the amodal 3D object detection problem for indoor scenarios. The solution is achieved through a novel neural network structure which takes a pair of RGB-D images as input and delivers oriented 3D bounding boxes as the output. Such network, named 3D-SSD, has two components: hierarchical feature fusion and multi-layer prediction. The hierarchical feature fusion combines multi-scale appearance and geometric features learned from RGB-D images, which is later utilized in the multi-layer prediction for object detection. Both the accuracy and the efficiency can be improved by exploiting 2.5D representations in a synergistic way. To specifically address the shape variance of different objects, a set of 3D anchor boxes with varying physical sizes are attached to every location on the prediction layers. While testing, the category scores for 3D anchor boxes are generated with adjusted positions, sizes and orientations, leading to the final detections using non-maximum suppression. Comprehensive experiments have been performed on publicly accessible dataset of SUN RGB-D and NYUV2. The results show the proposed algorithm is the first 3D detector that runs in near real-time on the challenging datasets with competitive performance to the state-of-the-art methods. The 3D-SSD gets 37.1% mAP on the SUN RGB-D dataset at around 5.6 fps, which outperforms the state-of-the-art Deep Sliding Shape by 10.2% mAP and around 109 x faster. For an efficient model setting with a rate of 9.3 fps, 3D-SSD still gets an accuracy of 37% on mAP. Further, experiments also suggest the proposed approach achieves comparable accuracy and is about 477 x faster than the state-of-art method on the NYUv2 dataset even with a smaller input image size. (C) 2019 Published by Elsevier B.V.

2D-Line-Drawing-Based 3D Object Recognition.

An Example-Based Approach to 3D Man-Made Object Reconstruction from Line Drawings

Separation of Line Drawings Based on Split Faces for 3D Object Reconstruction

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

A Line-Context Based Object Recognition Method

Geometric Object 3D Reconstruction from Single Line Drawing Image with Bottom-Up and Top-Down Classification and Sketch Generation.

Model based 3-D object recognition

Robust 3D Face Recognition by Local Shape Difference Boosting

Geometric Object 3D Reconstruction from Single Line Drawing Image Based on a Network for Classification and Sketch Extraction

3D Object Recognition By Corresponding and Quantizing Neural 3D Scene Representations

Scribble-Based 3D Shape Segmentation via Weakly-Supervised Learning

3D Object Reconstruction by Point Coordinates of 2D Line Drawings

A Shape Recognition Method Based on Graph- and Line-Contexts

3D Object Recognition System Based On Local Shape Descriptors and Depth Data Analysis

Cognitive Mechanism Related to Line Drawings and Its Applications in Intelligent Process of Visual Media: a Survey.

Solid Geometric Object Reconstruction from Single Line Drawing Image.

Efficient 2D-to-3D Correspondence Filtering for Scalable 3D Object Recognition

ObjectNet3D: A Large Scale Database for 3D Object Recognition

DGCB-Net: Dynamic Graph Convolutional Broad Network for 3D Object Recognition in Point Cloud

A PMJ-inspired Cognitive Framework for Natural Scene Categorization in Line Drawings.

3D Facial Expression Recognition Based on Properties of Line Segments Connecting Facial Feature Points