Abstract:LiDAR-based SLAM algorithms are extensively studied to providing robust and accurate positioning for autonomous driving vehicles (ADV) in the past decades. Satisfactory performance can be obtained using high-grade 3D LiDAR with 64 channels, which can provide dense point clouds. Unfortunately, the high price significantly prevents its extensive commercialization in ADV. The cost-effective 3D LiDAR with 16 channels is a promising replacement. However, only limited and sparse point clouds can be provided by the 16 channels LiDAR, which cannot guarantee sufficient positioning accuracy for ADV in challenging dynamic environments. The high-resolution image from the low-cost camera can provide ample information about the surroundings. However, the explicit depth information is not available from the image. Inspired by the complementariness of 3D LiDAR and camera, this paper proposes to make use of the high-resolution images from a camera to enrich the raw 3D point clouds from the low-cost 16 channels LiDAR based on a state-of-the-art deep learning algorithm. An ERFNet is firstly employed to segment the image with the aid of the raw sparse 3D point clouds. Meanwhile, the sparse convolutional neural network is employed to predict the dense point clouds based on raw sparse 3D point clouds. Then, the predicted dense point clouds are fused with the segmentation outputs from ERFnet using a novel multi-layer convolutional neural network to refine the predicted 3D point clouds. Finally, the enriched point clouds are employed to perform LiDAR SLAM based on the state-of-the-art normal distribution transform (NDT). We tested our approach on the re-edited KITTI datasets: (1)the sparse 3D point clouds are significantly enriched with a mean square error of 1.1m MSE. (2)the map generated from the LiDAR SLAM is denser which includes more details without significant accuracy loss.

80 × 120 AI-enhanced LiDAR System Based on a Lightweight Intensity–rgb–dtof Sensor Fusion Neural Network Deployed on an Edge Device

A 256 × 256 LiDAR Imaging System Based on a 200 mW SPAD-Based SoC with Microlens Array and Lightweight RGB-Guided Depth Completion Neural Network

228?×?304 200-mW lidar based on a single-point global-depth d-ToF sensor and RGB-guided super-resolution neural network

DELTAR: Depth Estimation from a Light-Weight ToF Sensor and RGB Image

A 40nm 2TOPS/W Depth-Completion Neural Network Accelerator SoC With Efficient Depth Engine for Realtime LiDAR Systems

LiDAR Data Enrichment Using Deep Learning Based on High-Resolution Image: An Approach to Achieve High-Performance LiDAR SLAM Using Low-cost LiDAR

Solid-State Dtof LiDAR System Using an Eight-Channel Addressable, 20w/ch Transmitter, and a 128x128 SPAD Receiver with SNR-Based Pixel Binning and Resolution Upscaling.

Construction of Edge Computing Platform Using 3D LiDAR and Camera Heterogeneous Sensing Fusion for Front Obstacle Recognition and Distance Measurement System

Robust single-photon 3D imaging based on full-scale feature integration and intensity edge guidance

Real-Time Stereo Image Depth Estimation Network with Group-Wise L1 Distance for Edge Devices Towards Autonomous Driving

An Optimized Deep Neural Network for Overhead Contact System Recognition from LiDAR Point Clouds

Dtof LIDAR System Using Addressable Multi-Channel VCSEL Transmitter, 128x80 SPAD Sensor, and ML-Based Object Detection for Adaptive Beam-Steering

A Novel CNN Architecture for Real-Time Point Cloud Recognition in Road Environment

LiDAR-based Outdoor Crowd Management for Smart Campus on the Edge

Fast Context-Awareness Encoder for LiDAR Point Semantic Segmentation

ImLiDAR: Cross-Sensor Dynamic Message Propagation Network for 3-D Object Detection

Real-time depth completion based on LiDAR-stereo for autonomous driving

Dynamic single-photon 3D imaging with a sparsity-based neural network

Real-Time Semantic Segmentation of LiDAR Point Clouds on Edge Devices for Unmanned Systems

Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles