Abstract:The need for a vehicle to perceive information about the external environmental as an independent intelligent individual has grown with the progress of intelligent driving from primary driver assistance to high-level autonomous driving. The ability of a common independent sensing unit to sense the external environment is limited by the sensor’s own characteristics and algorithm level. Hence, a common independent sensing unit fails to obtain comprehensive sensing information independently under conditions such as rain, fog, and night. Accordingly, an extended network-based fusion target detection algorithm for millimeter-wave radar and vision fusion is proposed in this work by combining the complementary perceptual performance of in-vehicle sensing elements, cost effectiveness, and maturity of independent detection technologies. Feature-level fusion is first used in this work according to the analysis of technical routes of the millimeter-wave radar and vision fusion. Training and test evaluation of the algorithm are carried out on the nuScenes dataset and test data from a homemade data acquisition platform. An extended investigation on the RetinaNet one-stage target detection algorithm based on the VGG-16+FPN backbone detection network is then conducted in this work to introduce millimeter-wave radar images as auxiliary information for visual image target detection. We use two-channel radar and three-channel visual images as inputs of the fusion network. We also propose an extended VGG-16 network applicable to millimeter-wave radar and visual fusion and an extended feature pyramid network. Test results showed that the mAP of the proposed network improves by 2.9% and the small target accuracy is enhanced by 18.73% compared with those of the reference network for pure visual image target detection. This finding verified the detection capability and algorithmic feasibility of the proposed extended fusion target detection network for visually insensitive targets.

A Novel Multi-Radar Plot Fusion Scheme Based on Parallel and Serial Plot Fusion Algorithm

Millimeter-Wave Radar and Vision Fusion Target Detection Algorithm Based on an Extended Network

A Novel Decision Fusion Scheme with Feedback in Neyman-Pearson Detection Systems.

An Improved Scheme for High-Resolution Point Cloud Map Generation Based on FMCW Radar

An Adaptive Fusion Algorithm for Multistatic and Multichannel Passive Radar Detection

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Performance analysis of serial data fusion algorithm under typical condition of multi-radar

A TLRTV Dual-band SAR Image Denoise-Fusion Strategy and Its Preliminary Experiment Analysis in Multi-band Airborne Radar System

A Distributed Detection Scheme for Multiple Monostatic Radars

Application of multi-sensor track fusion technology in fire control radar network

A Novel Data Fusion Algorithm to Combat False Data Injection Attacks in Networked Radar Systems

Research on Cooperative Detection Technology of Networked Radar Based on Data Fusion

Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving

MIMO Radar Signal Fusion Detection Method Based on Markov Chain

Target Tracking System for Multi-Sensor Data Fusion

A Graph-Based Track-Before-Detect Algorithm for Automotive Radar Target Detection

Fusion Technology of Multi-radar Target Based on VTS

Algorithm Fusion for 3D Ground-Penetrating Radar Imaging with Field Examples

Information Fusion for Radar Signal Sorting with the Distributed Reconnaissance Receivers

Multitarget Tracking Using Distributed Radar with Partially Overlap-ping Fields of Views

A Robust Target Detection Algorithm Based on the Fusion of Frequency-Modulated Continuous Wave Radar and a Monocular Camera