Abstract:In this work, we propose a novel one-stage and keypoint-based framework for monocular 3D object detection using only RGB images, called KM3D-Net. 2D detection only requires a deep neural network to predict 2D properties of objects, as it is a semanticity-aware task. For image-based 3D detection, we argue that the combination of a deep neural network and geometric constraints are needed to synergistically estimate appearance-related and spatial-related information. Here, we design a fully convolutional model to predict object keypoints, dimension, and orientation, and combine these with perspective geometry constraints to compute position attributes. Further, we reformulate the geometric constraints as a differentiable version and embed this in the network to reduce running time while maintaining the consistency of model outputs in an end-to-end fashion. Benefiting from this simple structure, we propose an effective semi-supervised training strategy for settings where labeled training data are scarce. In this strategy, we enforce a consensus prediction of two shared-weights KM3D-Net for the same unlabeled image under different input augmentation conditions and network regularization. In particular, we unify the coordinate-dependent augmentations as the affine transformation for the differential recovering position of objects, and propose a keypoint-dropout module for network regularization. Our model only requires RGB images, without synthetic data, instance segmentation, CAD model, or depth generator. Extensive experiments on the popular KITTI 3D detection dataset indicate that the KM3D-Net surpasses state-of-the-art methods by a large margin in both efficiency and accuracy. And also, to the best of our knowledge, this is the first application of semi-supervised learning in monocular 3D object detection. We surpass most of the previous fully supervised methods with only 13% labeled data on KITTI.

Person-MinkUNet: 3D Person Detection with LiDAR Point Cloud

Sparse Points to Dense Clouds: Enhancing 3D Detection with Limited LiDAR Data

Utilizing a Single-Stage 2D Detector in 3D LiDAR Point Cloud With Vertical Cylindrical Coordinate Projection for Human Identification

UniDet3D: Multi-dataset Indoor 3D Object Detection

M3D-RPN: Monocular 3D Region Proposal Network for Object Detection

2D vs. 3D LiDAR-based Person Detection on Mobile Robots

Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud

Efficient Detection and Tracking of Human Using 3D LiDAR Sensor

Monocular 3D Detection With Geometric Constraint Embedding and Semi-Supervised Training

MinkLoc3D-SI: 3D LiDAR Place Recognition With Sparse Convolutions, Spherical Coordinates, and Intensity

LU-Net: A Simple Approach to 3D LiDAR Point Cloud Semantic Segmentation

MonoNeRD: NeRF-like Representations for Monocular 3D Object Detection

LapUNet: a novel approach to monocular depth estimation using dynamic laplacian residual U-shape networks

MSL-Net: Sharp Feature Detection Network for 3D Point Clouds

SparseDet: A Simple and Effective Framework for Fully Sparse LiDAR-based 3D Object Detection

MSL3D: 3D object detection from monocular, stereo and point cloud for autonomous driving

Scanet: Spatial-Channel Attention Network For 3d Object Detection

3D-MiniNet: Learning a 2D Representation from Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation

MinkUNeXt: Point Cloud-based Large-scale Place Recognition using 3D Sparse Convolutions

Realtime Single-Shot Refinement Neural Network With Adaptive Receptive Field for 3D Object Detection From LiDAR Point Cloud

SIENet: Spatial Information Enhancement Network for 3D Object Detection from Point Cloud