Abstract:Purpose The purpose of this paper is the presentation and research of a novel sensor fusion-based system for obstacle detection and identification, which uses the millimeter-wave radar to detect the position and velocity of the obstacle. Afterwards, the image processing module uses the bounding box regression algorithm in deep learning to precisely locate and identify the obstacles. Design/methodology/approach Unlike the traditional algorithms that use radar and vision to detect obstacles separately, the purposed method of this paper uses radar to determine the approximate location of obstacles and then uses bounding box regression to achieve accurate positioning and recognition. First, the information of the obstacles can be acquired by the millimeter-wave radar, and the effective target is extracted by filtering the data. Then, use coordinate system conversion and camera parameter calibration to project the effective target to the image plane, and generate the region of interest (ROI). Finally, based on image processing and machine learning techniques, the vehicle targets in the ROI are detected and tracked. Findings The millimeter wave is used to determine the presence of an obstacle, and the deep learning algorithm of the image is combined to determine the shape and the class of the obstacle. The experimental results indicate that the detection rate of this method is up to 91.6 per cent, which can better implement the perception of the environment in front of the vehicle. Originality/value The originality is based on the combination of millimeter-wave sensors and deep learning. Using the bounding box regression algorithm in RCNN, the ROI detected by radar is analyzed to realize real-time obstacle detection and recognition. This method does not require processing the entire image, greatly reducing the amount of data processing and improving the efficiency of the algorithm.

Multi-layer Laser Radar Obstacle Detection in Complicated Environment

Obstacle Detection and Map Reconstruction Using Multi-Layer Laser Radar

Obstacle Detection Based on a Four-layer Laser Radar in Cross-country

Obstacle Detection by ALV Using Two 2D Laser Range Finders

Real-time dynamic obstacle detection and tracking using 3D Lidar

Robust vehicle detection using 3D Lidar under complex urban environment

Obstacle Avoidance Strategy with Path-following for USV Based on Obstacle-guidance Model Using 2D-Lidar in Urban River Conditions

Detection and Tracking of Campus Road Based on Four-Line Laser Radar

Monocular Vision Based Obstacle Detection for Robot Navigation in Unstructured Environment

Obstacle Detection for Autonomous Driving Vehicles With Multi-LiDAR Sensor Fusion

An Obstacle Detection and Avoidance System for Mobile Robot with a Laser Radar.

On-road multiple obstacles detection using color images and LiDAR point clouds

Environmental Modeling and Obstacle Avoidance of Mobile Robots Based on Laser Radar

Novel Laser-Based Obstacle Detection for Autonomous Robots on Unstructured Terrain

Obstacle detection for aircraft based on layered model

Radar And Vision Fusion For The Real-Time Obstacle Detection And Identification

Visual Navigation and Obstacle Avoidance Control for Agricultural Robots via LiDAR and Camera

Research on Obstacle Detection and Avoidance of Autonomous Underwater Vehicle Based on Forward-Looking Sonar

Monocular Camera-based Complex Obstacle Avoidance via Efficient Deep Reinforcement Learning

A Real-Time Multi-Constraints Obstacle Avoidance Method Using LiDAR

Obstacle Avoidance Using Range Data In Autonomous Navigation Of Mobile Robot