Abstract:Purpose The purpose of this paper is the presentation and research of a novel sensor fusion-based system for obstacle detection and identification, which uses the millimeter-wave radar to detect the position and velocity of the obstacle. Afterwards, the image processing module uses the bounding box regression algorithm in deep learning to precisely locate and identify the obstacles. Design/methodology/approach Unlike the traditional algorithms that use radar and vision to detect obstacles separately, the purposed method of this paper uses radar to determine the approximate location of obstacles and then uses bounding box regression to achieve accurate positioning and recognition. First, the information of the obstacles can be acquired by the millimeter-wave radar, and the effective target is extracted by filtering the data. Then, use coordinate system conversion and camera parameter calibration to project the effective target to the image plane, and generate the region of interest (ROI). Finally, based on image processing and machine learning techniques, the vehicle targets in the ROI are detected and tracked. Findings The millimeter wave is used to determine the presence of an obstacle, and the deep learning algorithm of the image is combined to determine the shape and the class of the obstacle. The experimental results indicate that the detection rate of this method is up to 91.6 per cent, which can better implement the perception of the environment in front of the vehicle. Originality/value The originality is based on the combination of millimeter-wave sensors and deep learning. Using the bounding box regression algorithm in RCNN, the ROI detected by radar is analyzed to realize real-time obstacle detection and recognition. This method does not require processing the entire image, greatly reducing the amount of data processing and improving the efficiency of the algorithm.

Design and implementation of a radar and camera-based obstacle classification system using machine-learning techniques

Obstacle Detection Based on a Four-layer Laser Radar in Cross-country

Obstacle Detection and Map Reconstruction Using Multi-Layer Laser Radar

Radar And Vision Fusion For The Real-Time Obstacle Detection And Identification

Machine Learning Approach for Obstacle Classification in Automotive Radar

Design of Multi-channel Ultrasonic Obstacle Avoidance Based on the Platform of Wheeled Intelligent Robot

Real-time obstacle detection by stereo vision and ultrasonic data fusion

Machine Learning Based Object Classification and Identification Scheme Using an Embedded Millimeter-Wave Radar Sensor

Advancing Cycling Safety: On-Bike Alert System Utilizing Multi-Layer Radar Point Cloud Clustering for Coarse Object Classification

Radar Signal Processing Architecture for Early Detection of Automotive Obstacles

Implementation of a perception system for autonomous vehicles using a detection-segmentation network in SoC FPGA

Design and Implementation of an Ultrasonic Sensor-Based Obstacle Avoidance System for Arduino Robots

Deep Learning Derived Object Detection and Tracking Technology Based on Sensor Fusion of Millimeter-Wave Radar/Video and Its Application on Embedded Systems

Obstacle Avoidance of Multi-Sensor Intelligent Robot Based on Road Sign Detection

A real-time obstacle avoidance and path tracking strategy for a mobile robot using machine-learning and vision-based approach

FPGA-Based Real-Time Object Detection and Classification System Using YOLO for Edge Computing

Obstacle Recognition Based on Machine Learning for On-Chip LiDAR Sensors in a Cyber-Physical System

An FPGA-Accelerated Design for Deep Learning Pedestrian Detection in Self-Driving Vehicles

HODET: Hybrid Object DEtection and Tracking using mmWave Radar and Visual Sensors

Evaluation of 3D Vulnerable Objects' Detection Using a Multi-Sensors System for Autonomous Vehicles

Research of Target Detection and Classification Techniques Using Millimeter-Wave Radar and Vision Sensors