Abstract:In this paper, we propose a weakly-supervised approach for 3D object detection, which makes it possible to train a strong 3D detector with position-level annotations (i.e. annotations of object centers and categories). In order to remedy the information loss from box annotations to centers, our method makes use of synthetic 3D shapes to convert the position-level annotations into virtual scenes with box-level annotations, and in turn utilizes the fully-annotated virtual scenes to complement the real labels. Specifically, we first present a shape-guided label-enhancement method, which assembles 3D shapes into physically reasonable virtual scenes according to the coarse scene layout extracted from position-level annotations. Then we transfer the information contained in the virtual scenes back to real ones by applying a virtual-to-real domain adaptation method, which refines the annotated object centers and additionally supervises the training of detector with the virtual scenes. Since the shape-guided label enhancement method generates virtual scenes by human-heuristic physical constraints, the layout of the fixed virtual scenes may be unreasonable with varied object combinations. To address this, we further present differentiable label enhancement to optimize the virtual scenes including object scales, orientations and locations in a data-driven manner. Moreover, we further propose a label-assisted self-training strategy to fully exploit the capability of detector. By reusing the position-level annotations and virtual scenes, we fuse the information from both domains and generate box-level pseudo labels on the real scenes, which enables us to directly train a detector in fully-supervised manner. Extensive experiments on the widely used ScanNet and Matterport3D datasets show that our approach surpasses current weakly-supervised and semi-supervised methods by a large margin, and achieves comparable detection performance with some popular fully-supervised methods with less than 5% of the labeling labor.

3D attention-driven depth acquisition for object identification.

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

Depth-aware gaze-following via auxiliary networks for robotics

Object-aware Guidance for Autonomous Scene Reconstruction

3D Object Detection with Attention: Shell-Based Modeling

3D Object Aided Self-Supervised Monocular Depth Estimation

Unsupervised Joint 3D Object Model Learning and 6D Pose Estimation for Depth-Based Instance Segmentation.

AutoShape: Real-Time Shape-Aware Monocular 3D Object Detection

AGO-Net: Association-Guided 3D Point Cloud Object Detection Network

Investigating Attention Mechanism in 3D Point Cloud Object Detection

Shape Prior Non-Uniform Sampling Guided Real-time Stereo 3D Object Detection

Fine-Grained 3D Shape Classification With Hierarchical Part-View Attention

DETR4D: Direct Multi-View 3D Object Detection with Sparse Attention

Back to Reality: Learning Data-Efficient 3D Object Detector with Shape Guidance.

Neural Attention Field: Emerging Point Relevance in 3D Scenes for One-Shot Dexterous Grasping

3D Object Recognition in Cluttered Scenes with Robust Shape Description and Correspondence Selection.

Deep learning‐based object identification with instance segmentation and pseudo‐LiDAR point cloud for work zone safety

KDA3D: Key-Point Densification and Multi-Attention Guidance for 3D Object Detection

Scanet: Spatial-Channel Attention Network For 3d Object Detection

Probabilistic Instance Shape Reconstruction with Sparse LiDAR for Monocular 3D Object Detection

3D Object Recognition By Corresponding and Quantizing Neural 3D Scene Representations