Abstract:LiDAR-based sparse 3D object detection plays a crucial role in autonomous driving applications due to its computational efficiency advantages. Existing methods either use the features of a single central voxel as an object proxy, or treat an aggregated cluster of foreground points as an object proxy. However, the former lacks the ability to aggregate contextual information, resulting in insufficient information expression in object proxies. The latter relies on multi-stage pipelines and auxiliary tasks, which reduce the inference speed. To maintain the efficiency of the sparse framework while fully aggregating contextual information, in this work, we propose SparseDet which designs sparse queries as object proxies. It introduces two key modules, the Local Multi-scale Feature Aggregation (LMFA) module and the Global Feature Aggregation (GFA) module, aiming to fully capture the contextual information, thereby enhancing the ability of the proxies to represent objects. Where LMFA sub-module achieves feature fusion across different scales for sparse key voxels %which does this through via coordinate transformations and using nearest neighbor relationships to capture object-level details and local contextual information, GFA sub-module uses self-attention mechanisms to selectively aggregate the features of the key voxels across the entire scene for capturing scene-level contextual information. Experiments on nuScenes and KITTI demonstrate the effectiveness of our method. Specifically, on nuScene, SparseDet surpasses the previous best sparse detector VoxelNeXt by 2.2\% mAP with 13.5 FPS, and on KITTI, it surpasses VoxelNeXt by 1.12\% $\mathbf{AP_{3D}}$ on hard level tasks with 17.9 FPS.

VS-Net: A Voxel Encoding and Sparse Convolution Embedded Network for LiDAR 3D Object Detection.

SSF: Sparse Point Cloud Object Detection Based on Self-Adaptive Voxel Encoding and Focal-Sparse Convolution

SP-Net: A Sparse Convolution and Point-Encoding Enhanced Network for 3D Object Detection in LiDAR Point Clouds.

VP-Net: Voxels As Points for 3-D Object Detection.

VP-Net: Voxels as Points for 3D Object Detection

VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking

Voxel self-attention and center-point for 3D object detector

VSL-Net: Voxel structure learning for 3D object detection

VoxelFSD: voxel-based fully sparse detector with sparse convolution for 3D object detection

DVFENet: Dual-branch voxel feature extraction network for 3D object detection

PVConvNet: Pixel-Voxel Sparse Convolution for multimodal 3D object detection

Voxel or Pillar: Exploring Efficient Point Cloud Representation for 3D Object Detection

SparseDet: A Simple and Effective Framework for Fully Sparse LiDAR-based 3D Object Detection

SVGA-Net: Sparse Voxel-Graph Attention Network for 3D Object Detection from Point Clouds

VIN: Voxel-based Implicit Network for Joint 3D Object Detection and Segmentation for Lidars

SIEV-Net: A Structure-Information Enhanced Voxel Network for 3D Object Detection From LiDAR Point Clouds

Real-Time 3D Object Detection From Point Cloud Through Foreground Segmentation

Virtual Sparse Convolution for Multimodal 3D Object Detection

Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection

An Effective Voxelization Method for LiDAR-based 3D Object Detection

BEV-Net: A Bird’s Eye View Object Detection Network for LiDAR Point Cloud