Abstract:The rapid development of Autonomous Vehicles (AVs) increases the requirement for the accurate prediction of objects in the vicinity to guarantee safer journeys. For effectively predicting objects, sensors such as Three-Dimensional Light Detection and Ranging (3D LiDAR) and cameras can be used. The 3D LiDAR sensor captures the 3D shape of the object and produces point cloud data that describes the geometrical structure of the object. The LiDAR-only detectors may be subject to false detection or even non-detection over objects located at high distances. The camera sensor captures RGB images with sufficient attributes that describe the distinct identification of the object. The high-resolution images produced by the camera sensor benefit the precise classification of the objects. However, hindrances such as the absence of depth information from the images, unstructured point clouds, and cross modalities affect assertion and boil down the environmental perception. To this end, this paper proposes an object detection mechanism that fuses the data received from the camera sensor and the 3D LiDAR sensor (OD-C3DL). The 3D LiDAR sensor obtains point clouds of the object such as distance, position, and geometric shape. The OD-C3DL employs Convolutional Neural Networks (CNN) for further processing point clouds obtained from the 3D LiDAR sensor and the camera sensor to recognize the objects effectively. The point cloud of the LiDAR is enhanced and fused with the image space on the Regions of Interest (ROI) for easy recognition of the objects. The evaluation results show that the OD-C3DL can provide an average of 89 real-time objects for a frame and reduces the extraction time by a recall rate of 94%. The average processing time is 65ms, which makes the OD-C3DL model incredibly suitable for the AVs perception. Furthermore, OD-C3DL provides mean accuracy for identifying automobiles and pedestrians at a moderate degree of difficulty is higher than that of the previous models at 79.13% and 88.76%.

A Real-Time Distance Prediction via Deep Learning and Microsoft Kinect

IR Stereo RealSense: Decreasing Minimum Range of Navigational Assistance for Visually Impaired Individuals.

Depth Estimation of Traffic Scenes from Image Sequence Using Deep Learning.

KineDepth: Utilizing Robot Kinematics for Online Metric Depth Estimation

Improving RealSense by Fusing Color Stereo Vision and Infrared Stereo Vision for the Visually Impaired

Real Life Human Movement Realization in Multimodal Group Communication Using Depth Map Information and Machine Learning

DELTAR: Depth Estimation from a Light-Weight ToF Sensor and RGB Image

Defogging Kinect: Simultaneous Estimation of Object Region and Depth in Foggy Scenes

3D Vehicle Detection Using Cheap LiDAR and Camera Sensors.

Depth-color fusion strategy for 3-D scene modeling with Kinect

Enhancing Visual Odometry with Estimated Scene Depth: Leveraging RGB-D Data with Deep Learning

Self-supervised Depth Estimation Leveraging Global Perception and Geometric Smoothness Using On-board Videos

FusionMapping: Learning Depth Prediction with Monocular Images and 2D Laser Scans

Real-time 3D human objects rendering based on multiple camera details

Real-Time 3D Object Detection and Classification in Autonomous Driving Environment Using 3D LiDAR and Camera Sensors

Real-time 3d reconstruction of dynamic scenes with multiple kinect v2 sensors

An Object Perception and Positioning Method Via Deep Perception Learning Object Detection

Real-time human detection and tracking in complex environments using single RGBD camera

DECADE: Towards Designing Efficient-yet-Accurate Distance Estimation Modules for Collision Avoidance in Mobile Advanced Driver Assistance Systems

A High Spatial Resolution Depth Sensing Method Based on Binocular Structured Light

Real-Time Stereo Image Depth Estimation Network with Group-Wise L1 Distance for Edge Devices Towards Autonomous Driving