Abstract:LiDAR-based sparse 3D object detection plays a crucial role in autonomous driving applications due to its computational efficiency advantages. Existing methods either use the features of a single central voxel as an object proxy, or treat an aggregated cluster of foreground points as an object proxy. However, the former lacks the ability to aggregate contextual information, resulting in insufficient information expression in object proxies. The latter relies on multi-stage pipelines and auxiliary tasks, which reduce the inference speed. To maintain the efficiency of the sparse framework while fully aggregating contextual information, in this work, we propose SparseDet which designs sparse queries as object proxies. It introduces two key modules, the Local Multi-scale Feature Aggregation (LMFA) module and the Global Feature Aggregation (GFA) module, aiming to fully capture the contextual information, thereby enhancing the ability of the proxies to represent objects. Where LMFA sub-module achieves feature fusion across different scales for sparse key voxels %which does this through via coordinate transformations and using nearest neighbor relationships to capture object-level details and local contextual information, GFA sub-module uses self-attention mechanisms to selectively aggregate the features of the key voxels across the entire scene for capturing scene-level contextual information. Experiments on nuScenes and KITTI demonstrate the effectiveness of our method. Specifically, on nuScene, SparseDet surpasses the previous best sparse detector VoxelNeXt by 2.2\% mAP with 13.5 FPS, and on KITTI, it surpasses VoxelNeXt by 1.12\% $\mathbf{AP_{3D}}$ on hard level tasks with 17.9 FPS.

Advancing Point Cloud Detection With Sparse Intensity Encoder: A Novel Voxelization Approach

SSF: Sparse Point Cloud Object Detection Based on Self-Adaptive Voxel Encoding and Focal-Sparse Convolution

SIEV-Net: A Structure-Information Enhanced Voxel Network for 3D Object Detection From LiDAR Point Clouds

Voxel or Pillar: Exploring Efficient Point Cloud Representation for 3D Object Detection

VFL3D: A Single-Stage Fine-Grained Lightweight Point Cloud 3D Object Detection Algorithm Based on Voxels

3D Point Cloud Object Detection Method Based on Multi-Scale Dynamic Sparse Voxelization

Real-Time Point Cloud Object Detection via Voxel-Point Geometry Abstraction

PV-RCNN++: Semantical Point-Voxel Feature Interaction for 3D Object Detection

VS-Net: A Voxel Encoding and Sparse Convolution Embedded Network for LiDAR 3D Object Detection.

VP-Net: Voxels as Points for 3D Object Detection

An Effective Voxelization Method for LiDAR-based 3D Object Detection

VoxelNextFusion: A Simple, Unified and Effective Voxel Fusion Framework for Multi-Modal 3D Object Detection

SparseDet: A Simple and Effective Framework for Fully Sparse LiDAR-based 3D Object Detection

Semantic-aware 3D-voxel CenterNet for point cloud object detection

Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion

Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds

DV-Det: Efficient 3D Point Cloud Object Detection with Dynamic Voxelization

Spatial Information Enhancement with Multi-Scale Feature Aggregation for Long-Range Object and Small Reflective Area Object Detection from Point Cloud

V2P-SSD: Single-Stage 3-D Object Detection With Voxel-to-Point Transformation

VIC-Net: Voxelization Information Compensation Network for Point Cloud 3D Object Detection

PV-SSD: A Multi-Modal Point Cloud Feature Fusion Method for Projection Features and Variable Receptive Field Voxel Features