Abstract:Purpose The purpose of this paper is the presentation and research of a novel sensor fusion-based system for obstacle detection and identification, which uses the millimeter-wave radar to detect the position and velocity of the obstacle. Afterwards, the image processing module uses the bounding box regression algorithm in deep learning to precisely locate and identify the obstacles. Design/methodology/approach Unlike the traditional algorithms that use radar and vision to detect obstacles separately, the purposed method of this paper uses radar to determine the approximate location of obstacles and then uses bounding box regression to achieve accurate positioning and recognition. First, the information of the obstacles can be acquired by the millimeter-wave radar, and the effective target is extracted by filtering the data. Then, use coordinate system conversion and camera parameter calibration to project the effective target to the image plane, and generate the region of interest (ROI). Finally, based on image processing and machine learning techniques, the vehicle targets in the ROI are detected and tracked. Findings The millimeter wave is used to determine the presence of an obstacle, and the deep learning algorithm of the image is combined to determine the shape and the class of the obstacle. The experimental results indicate that the detection rate of this method is up to 91.6 per cent, which can better implement the perception of the environment in front of the vehicle. Originality/value The originality is based on the combination of millimeter-wave sensors and deep learning. Using the bounding box regression algorithm in RCNN, the ROI detected by radar is analyzed to realize real-time obstacle detection and recognition. This method does not require processing the entire image, greatly reducing the amount of data processing and improving the efficiency of the algorithm.

Deep Learning Based Smart Radar Vision System for Object Recognition

Automatic Target Recognition of Millimeter-Wave Radar Based on Deep Learning

Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors

Research of Target Detection and Classification Techniques Using Millimeter-Wave Radar and Vision Sensors

300 GHz Radar Object Recognition based on Deep Neural Networks and Transfer Learning

Fast Detection and Recognition Method of UAV in Sky Background

Radar And Vision Fusion For The Real-Time Obstacle Detection And Identification

A Systematic Study on Object Recognition Using Millimeter-wave Radar

Point-wise Discriminative Auto-Encoder with Application on Robust Radar Automatic Target Recognition.

Obstacle Recognition for Intelligent Vehicle Based on Radar and Vision Fusion.

Target detection and recognition of radar spectrum image based on deep learning

Object detection and recognition of intelligent service robot based on deep learning

Object Detection of Millimeter Wave Radar Based on Computer Vision

Traffic Target Recognition Method Based on Millimeter-Wave Radar

An Obstacle Detection System for a Mobile Robot Based on Radar-Vision Fusion

Intelligent Radar Jamming Recognition in Open Set Environment Based on Deep Learning Networks

DEEP LEARNING FOR OBJECT DETECTION USING RADAR DATA

mmWave-YOLO: A mmWave Imaging Radar-Based Real-Time Multiclass Object Recognition System for ADAS Applications

Radar Object Detection Using 3D-2D Encoder-Decoder Network

Improving Recognition of Road Users via Doppler Radar Data and Deep Learning Convolutional Networks