Abstract:Accurately and reliably perceiving the environment is a major challenge in autonomous driving and robotics research. Traditional vision-based methods often suffer from varying lighting conditions, occlusions, and complex environments. This paper addresses these challenges by combining a deep learning-based object detection algorithm, YOLOv8, with LiDAR data fusion technology. The principle of this combination is to merge the advantages of these technologies: YOLOv8 excels in real-time object detection and classification through RGB images, while LiDAR provides accurate distance measurement and 3D spatial information, regardless of lighting conditions. The integration aims to apply the high accuracy and robustness of YOLOv8 in identifying and classifying objects, as well as the depth data provided by LiDAR. This combination enhances the overall environmental perception, which is critical for the reliability and safety of autonomous systems. However, this fusion brings some research challenges, including data calibration between different sensors, filtering ground points from LiDAR point clouds, and managing the computational complexity of processing large datasets. This paper presents a comprehensive approach to address these challenges. Firstly, a simple algorithm is introduced to filter out ground points from LiDAR point clouds, which are essential for accurate object detection, by setting different threshold heights based on the terrain. Secondly, YOLOv8, trained on a customized dataset, is utilized for object detection in images, generating 2D bounding boxes around detected objects. Thirdly, a calibration algorithm is developed to transform 3D LiDAR coordinates to image pixel coordinates, which are vital for correlating LiDAR data with image-based object detection results. Fourthly, a method for clustering different objects based on the fused data is proposed, followed by an object tracking algorithm to compute the 3D poses of objects and their relative distances from a robot. The Agilex Scout Mini robot, equipped with Velodyne 16-channel LiDAR and an Intel D435 camera, is employed for data collection and experimentation. Finally, the experimental results validate the effectiveness of the proposed algorithms and methods.

YOLO based Object Detection Techniques for Autonomous Driving

ObjectDetect: A Real-Time Object Detection Framework for Advanced Driver Assistant Systems Using YOLOv5

Object Detection for Autonomous Driving using YOLO algorithm.

Small-Object Detection based on YOLOv5 in Autonomous Driving Systems

Object Detection for Self-Driving Car in Complex Traffic Scenarios

Object Detection and Classification Framework for Analysis of Video Data Acquired from Indian Roads

Real Time Object Detection System Using Yolov

Object Detection in Specific Traffic Scenes using YOLOv2

MobileYOLO: Real-Time Object Detection Algorithm in Autonomous Driving Scenarios

DL-YOLOX: Real-time object detection via adjustable dilated enhancement for autonomous driving scene

Real-Time Object Detection Based on YOLO-v2 for Tiny Vehicle Object

YOLOv8-QSD: An Improved Small Object Detection Algorithm for Autonomous Vehicles Based on YOLOv8

YOLO-based Object Detection Models: A Review and its Applications

An Improved Traffic Sign Recognition and Road Lane Detection for Self-Driving Cars using YOLO-v8

YOLO-Based Vehicle Detection: Literature Review

YOLO-Vehicle-Pro: A Cloud-Edge Collaborative Framework for Object Detection in Autonomous Driving under Adverse Weather Conditions

Application of improved you only look once model in road traffic monitoring system

Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles

An Advanced Approach to Object Detection and Tracking in Robotics and Autonomous Vehicles Using YOLOv8 and LiDAR Data Fusion

A Real-Time Object Detector for Autonomous Vehicles Based on YOLOv4

Assessing the YOLO Series Through Empirical Analysis on the KITTI Dataset for Autonomous Driving