Abstract:Purpose The purpose of this paper is the presentation and research of a novel sensor fusion-based system for obstacle detection and identification, which uses the millimeter-wave radar to detect the position and velocity of the obstacle. Afterwards, the image processing module uses the bounding box regression algorithm in deep learning to precisely locate and identify the obstacles. Design/methodology/approach Unlike the traditional algorithms that use radar and vision to detect obstacles separately, the purposed method of this paper uses radar to determine the approximate location of obstacles and then uses bounding box regression to achieve accurate positioning and recognition. First, the information of the obstacles can be acquired by the millimeter-wave radar, and the effective target is extracted by filtering the data. Then, use coordinate system conversion and camera parameter calibration to project the effective target to the image plane, and generate the region of interest (ROI). Finally, based on image processing and machine learning techniques, the vehicle targets in the ROI are detected and tracked. Findings The millimeter wave is used to determine the presence of an obstacle, and the deep learning algorithm of the image is combined to determine the shape and the class of the obstacle. The experimental results indicate that the detection rate of this method is up to 91.6 per cent, which can better implement the perception of the environment in front of the vehicle. Originality/value The originality is based on the combination of millimeter-wave sensors and deep learning. Using the bounding box regression algorithm in RCNN, the ROI detected by radar is analyzed to realize real-time obstacle detection and recognition. This method does not require processing the entire image, greatly reducing the amount of data processing and improving the efficiency of the algorithm.

Radar and Camera Fusion Based Moving Obstacle Tracking for Automated Vehicles

Online Multipedestrian Tracking Based on Fused Detections of Millimeter Wave Radar and Vision

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

Assisting the Visually Impaired: Multitarget Warning Through Millimeter Wave Radar and RGB-depth Sensors.

Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving

Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

Fusion Perception of Vision and Millimeter Wave Radar for Autonomous Driving

3D Multiple Object Tracking with Multi-modal Fusion of Low-cost Sensors for Autonomous Driving.

Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera

Radar And Vision Fusion For The Real-Time Obstacle Detection And Identification

Multi-sensor Information Fusion for Unmanned Cars Using Radar Map

Driving Assistance System Based on Data Fusion of Multisource Sensors for Autonomous Unmanned Ground Vehicles.

Research of Target Detection and Classification Techniques Using Millimeter-Wave Radar and Vision Sensors

Robust Target Recognition and Tracking of Self-Driving Cars With Radar and Camera Information Fusion Under Severe Weather Conditions

Multi-Sensor Fusion on Automated Guided Vehicle

Multi-Camera Object Fusion Tracking Model for Autonomous Driving.

Camera-Radar Fusion Sensing System Based on Multi-Layer Perceptron

Multi-sensor Fusion Detection Method for Vehicle Target Based on Kalman Filter and Data Association Filter

An Interactive Multi Model prediction-based Radar-Camera Fusion Algorithm for Autonomous Vehicles