Abstract:Voxel-based 3D object detection has been widely applied in robotics, virtual reality, and autonomous driving. However, inefficiency in the voxelization and backbone-network computation, which are the main components of voxel-based models, prevents efficient 3D object detection. First, due to the high sparsity and irregularity of the point cloud, the voxelization process usually requires generalized platforms, such as CPUs, and causes low voxelization speed. Second, the voxel-based models contain considerable transposed convolutional layers, and existing accelerators introduce considerable additional hardware to support both convolution and transposed convolution operations. Nonetheless, this strategy incurs significant hardware costs. Besides, transposed convolutions result in various patterns of sparse feature maps, and pruning, as a representative model compression technique, results in sparse weight matrices. The two types of sparsity impose challenges in accelerating the voxel-based models, including activation-weight matching efficiency, low partial-sum accumulation efficiency, and workload imbalance issues. In this work, we propose DSAV, a 3D object detection accelerator to address these obstacles. Specifically, we first propose a hash-based voxelizer for efficient voxelization, by storing and indexing voxels hierarchically. Then, we collaboratively design the transposed convolution acceleration method, structured pruning method, and accelerator architecture for the voxel-based models. As a result, the accelerator can fully leverage the sparsity lies in both feature maps and weight matrices. Experimental results show that the proposed accelerator can outperform prior studies by 19× ~ 19.8× faster in voxelization, and 4.29× ~38.01× faster in backbone inference. Finally, the accelerator achieves 4.61× ~ 31.63× speedups than its counterparts in 3D object detection tasks.

A Feature Extraction Framework for 3D Scientific Voxel Object Using SVD and Neural Network

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

DVFENet: Dual-branch voxel feature extraction network for 3D object detection

Voxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Image

VoxelFSD: voxel-based fully sparse detector with sparse convolution for 3D object detection

DSAV: A Deep Sparse Acceleration Framework for Voxel-Based 3D Object Detection

Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection

SparseVoxNet: 3-D Object Recognition With Sparsely Aggregation of 3-D Dense Blocks

A voxelized point clouds representation for object classification and segmentation on 3D data

3D Voxel Reconstruction from Single-View Image Based on Cross-Domain Feature Fusion

PV-SSD: A Multi-Modal Point Cloud Feature Fusion Method for Projection Features and Variable Receptive Field Voxel Features

MsSVT++: Mixed-scale Sparse Voxel Transformer with Center Voting for 3D Object Detection

VoxelEmbed: 3D Instance Segmentation and Tracking with Voxel Embedding based Deep Learning

A High-Quality Voxel 3D Reconstruction System for Large Scenes Based on the Branch and Bound Method

Novel 3D local feature descriptor of point clouds based on spatial voxel homogenization for feature matching

Slot-guided Volumetric Object Radiance Fields

Multi-scale Latent Feature-Aware Network for Logical Partition Based 3D Voxel Reconstruction

A Study of 3D model voxelization method for artificial intelligence learning

PointVoxelFormer -- Reviving point cloud networks for 3D medical imaging

PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection

VV-Net: Voxel VAE Net with Group Convolutions for Point Cloud Segmentation