Abstract:Robust sensor fusion is a key technology for enabling the safe operation of automated vehicles. Sensor fusion typically utilizes inputs of cameras, radars, lidar, inertial measurement unit, and global navigation satellite systems, process them, and then output object detection or positioning data. This paper will focus on sensor fusion between the camera, radar, and vehicle wheel speed sensors which is a critical need for near-term realization of sensor fusion benefits. The camera is an off-the-shelf computer vision product from MobilEye and the radar is a Delphi/Aptive electronically scanning radar (ESR) both of which are connected to a drive-by-wire capable vehicle platform. We utilize the MobilEye and wheel speed sensors to create a dynamic region of interest (DROI) of the drivable region that changes as the vehicle moves through the environment. The use of the DROI can reduce the need for up to approximately 100% of the detections from radar, for processing of the driveable region. This provides not only accurate and robust detections but also has benefits in lowering computational power and time complexity. We then continue to reduce the number of detections in the driveable region using machine learning techniques such as density-based spatial clustering and applications with noise (DBSCAN). This is followed by KMeans clustering to further reduce detections and lastly fused with the extended Kalman filter. Our experimental results obtained using an instrumented vehicle show a large reduction in the need for radar detections processing after both, fusion with our DROI and further clustering using machine learning techniques for the driveable region. Our proposed complete technique decreases the amount of fused misdetection, decreases computational power, and increases the reliability of the fused perception model which can greatly benefit current Advanced Driver Assistance System products available on the market today.

Real-Time Bird’s Eye Surround View System: An Embedded Perspective

3-D Surround View for Advanced Driver Assistance Systems.

Intelligent Driver Assistance System Based on Panoramic Video Technique

Capture Of Vehicle Surroundings Using A Pair Of Panoramic Annular Lens Cameras

Real-time Detection, Tracking, and Classification of Moving and Stationary Objects using Multiple Fisheye Images

Surround-View Fisheye Camera Perception for Automated Driving: Overview, Survey & Challenges

Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges

Deep Perspective Transformation Based Vehicle Localization on Bird's Eye View

Disentangling and Vectorization: A 3D Visual Perception Approach for Autonomous Driving Based on Surround-View Fisheye Cameras

Vehicle Video Surveillance System Based on Image Fusion and Parallel Computing

Panoramic visual system for spherical mobile robots

Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges

A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird's Eye View

Enhanced Parking Perception by Multi-Task Fisheye Cross-view Transformers

Real-time Full-stack Traffic Scene Perception for Autonomous Driving with Roadside Cameras

An Event-Driven Stereo System for Real-Time 3-D 360° Panoramic Vision

Real-time Multi-view Omnidirectional Depth Estimation System for Robots and Autonomous Driving on Real Scenes

YOLO-BEV: Generating Bird's-Eye View in the Same Way as 2D Object Detection

A Practical Panorama Parking Assistant System Based on Image Splicing

Higher Accuracy and Lower Computational Perception Environment Based Upon a Real-time Dynamic Region of Interest

Understanding Bird's-Eye View of Road Semantics using an Onboard Camera