Abstract:This paper aims at developing a faster and more accurate solution to the amodal 3D object detection problem for indoor scenarios. The solution is achieved through a novel neural network structure which takes a pair of RGB-D images as input and delivers oriented 3D bounding boxes as the output. Such network, named 3D-SSD, has two components: hierarchical feature fusion and multi-layer prediction. The hierarchical feature fusion combines multi-scale appearance and geometric features learned from RGB-D images, which is later utilized in the multi-layer prediction for object detection. Both the accuracy and the efficiency can be improved by exploiting 2.5D representations in a synergistic way. To specifically address the shape variance of different objects, a set of 3D anchor boxes with varying physical sizes are attached to every location on the prediction layers. While testing, the category scores for 3D anchor boxes are generated with adjusted positions, sizes and orientations, leading to the final detections using non-maximum suppression. Comprehensive experiments have been performed on publicly accessible dataset of SUN RGB-D and NYUV2. The results show the proposed algorithm is the first 3D detector that runs in near real-time on the challenging datasets with competitive performance to the state-of-the-art methods. The 3D-SSD gets 37.1% mAP on the SUN RGB-D dataset at around 5.6 fps, which outperforms the state-of-the-art Deep Sliding Shape by 10.2% mAP and around 109 x faster. For an efficient model setting with a rate of 9.3 fps, 3D-SSD still gets an accuracy of 37% on mAP. Further, experiments also suggest the proposed approach achieves comparable accuracy and is about 477 x faster than the state-of-art method on the NYUv2 dataset even with a smaller input image size. (C) 2019 Published by Elsevier B.V.

Part Detection for 3D Shapes via Multi-view Rendering.

Region-based Saliency Estimation for 3D Shape Analysis and Understanding

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

Unify 3D Shape Retrieval and Classification in One Framework

A Multi-view 3D Vehicle Detection Method Based On Novel 3D Proposal Generation Method

A Unified Feature Representation and Learning Framework for 3D Shape

3D Shape Segmentation Based on Viewpoint Entropy and Projective Fully Convolutional Networks Fusing Multi-view Features

Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views

Co-segmentation of 3D Shapes Via Multi-View Spectral Clustering

Pose and Shape Estimation with Discriminatively Learned Parts

SPV-SSD: An Anchor-Free 3D Single-Stage Detector with Supervised-PointRendering and Visibility Representation

Part123: Part-aware 3D Reconstruction from a Single-view Image

3M3D: Multi-view, Multi-path, Multi-representation for 3D Object Detection

PREMA: Part-based REcurrent Multi-view Aggregation Network for 3D Shape Retrieval

Unsupervised Detection of Distinctive Regions on 3D Shapes

A Multi-Modal Attention-Based Approach for Points of Interest Detection on 3D Shapes

Shape Topics: A Compact Representation and New Algorithms for 3D Partial Shape Retrieval

Sketch-Based 3D Shape Retrieval with Multi-View Fusion Transformer

Learning Fine-Grained Segmentation of 3D Shapes without Part Labels

DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

From Points to Multi-Object 3D Reconstruction