Abstract:As Light Detection And Ranging (LiDAR) increasingly becomes an essential component in robotic navigation and autonomous driving, the processing of high throughput 3D point clouds in real time is widely required. This work considers the point cloud k-Nearest Neighbor (kNN) search, which is an important 3D processing kernel. Although applying fine-grained parallelism optimization on internal processing, e.g., using multiple workers, has demonstrated high efficiency, previous accelerators with DDR external memory are fundamentally limited by the external bandwidth bottleneck. To break this bottleneck, this work proposes a highly parallel architecture, namely ParallelNN, for highly efficient kNN search processing of high throughput point clouds. First, we optimize the multichannel cache based on High Bandwidth Memory (HBM) and on-chip memory to provide large external bandwidth. Then, a novel parallel depth-first octree construction algorithm is proposed and mapped onto multiple construction branches with trace-coded construction queues, which can regularize random accesses and perform multi-branch octree construction efficiently. Furthermore, in the search stage, we present algorithm-architecture co-optimization, including parallel keyframe-based scheduling and multi-branch flexible search engines, to provide conflict-free access and maximum reuse opportunities for reference points, which achieves more than 27.0× speedup compared with baseline architectures. We prototype ParallelNN on Virtex HBM FPGA and perform extensive benchmarking on the KITTI dataset. The results demonstrate that ParallelNN achieves up to 107.7× and 12.1× speedup over CPU and GPU implementations, while being more energy efficient, e.g., outperforming CPU and GPU implementations by 73.6× and 31.1×, respectively. Besides, with the proposed algorithm-architecture co-optimization, ParallelNN achieves 11.4× speedup over state-of-the-art architecture. Moreover, ParallelNN is configurable and can be easily generalized to similar octree-based applications.

A Demonstration Platform for Large-Scaled Point Cloud Network Based on 28nm 2D/3D Unified Sparse Convolution Accelerator.

A 28nm 2D/3D Unified Sparse Convolution Accelerator with Block-Wise Neighbor Searcher for Large-Scaled Voxel-Based Point Cloud Network.

A 28-Nm Energy-Efficient Sparse Neural Network Processor for Point Cloud Applications Using Block-Wise Online Neighbor Searching

An Efficient FPGA Accelerator for Point Cloud

Pass3d: Precise And Accelerated Semantic Segmentation For 3d Point Cloud

DaDianNao: A Machine-Learning Supercomputer

A Low-Latency Framework with Algorithm-Hardware Co-Optimization for 3-D Point Cloud

Accelerating DNN-based 3D point cloud processing for mobile computing

SpCNA: an FPGA-based Accelerator for Point Cloud Convolutional Neural Networks.

Voxel-CIM: An Efficient Compute-in-Memory Accelerator for Voxel-based Point Cloud Neural Networks

SpOctA: A 3D Sparse Convolution Accelerator with Octree-Encoding-Based Map Search and Inherent Sparsity-Aware Processing

AccSS3D: Accelerator for Spatially Sparse 3D DNNs

Balancing Cpu And Gpu: Real-Time Visualization Of Large Scale 3d Scanning Models

ParallelNN: A Parallel Octree-based Nearest Neighbor Search Accelerator for 3D Point Clouds

A-U3D: A Unified 2D/3D CNN Accelerator on the Versal Platform for Disparity Estimation.

PCSCNet: Fast 3D Semantic Segmentation of LiDAR Point Cloud for Autonomous Car using Point Convolution and Sparse Convolution Network

SimDiff: Point Cloud Acceleration by Utilizing Spatial Similarity and Differential Execution

Real-Time Semantic Segmentation of Point Clouds Based on an Attention Mechanism and a Sparse Tensor

DPC-Net: Distributed Point Convolution Network for large-scale point clouds semantic segmentation

7.2 A 12nm Programmable Convolution-Efficient Neural-Processing-Unit Chip Achieving 825TOPS

An Efficient Accelerator for Point-based and Voxel-based Point Cloud Neural Networks