Abstract:The need for a vehicle to perceive information about the external environmental as an independent intelligent individual has grown with the progress of intelligent driving from primary driver assistance to high-level autonomous driving. The ability of a common independent sensing unit to sense the external environment is limited by the sensor’s own characteristics and algorithm level. Hence, a common independent sensing unit fails to obtain comprehensive sensing information independently under conditions such as rain, fog, and night. Accordingly, an extended network-based fusion target detection algorithm for millimeter-wave radar and vision fusion is proposed in this work by combining the complementary perceptual performance of in-vehicle sensing elements, cost effectiveness, and maturity of independent detection technologies. Feature-level fusion is first used in this work according to the analysis of technical routes of the millimeter-wave radar and vision fusion. Training and test evaluation of the algorithm are carried out on the nuScenes dataset and test data from a homemade data acquisition platform. An extended investigation on the RetinaNet one-stage target detection algorithm based on the VGG-16+FPN backbone detection network is then conducted in this work to introduce millimeter-wave radar images as auxiliary information for visual image target detection. We use two-channel radar and three-channel visual images as inputs of the fusion network. We also propose an extended VGG-16 network applicable to millimeter-wave radar and visual fusion and an extended feature pyramid network. Test results showed that the mAP of the proposed network improves by 2.9% and the small target accuracy is enhanced by 18.73% compared with those of the reference network for pure visual image target detection. This finding verified the detection capability and algorithmic feasibility of the proposed extended fusion target detection network for visually insensitive targets.

Target detection with multi-sensor networks:a survey

Distributed Sensor Layout Optimization for Target Detection with Data Fusion.

A Novel Decision Fusion Scheme with Feedback in Neyman-Pearson Detection Systems.

Millimeter-Wave Radar and Vision Fusion Target Detection Algorithm Based on an Extended Network

Detection, tracking and recognition of multi-targets using fusion data from different sensors

Target Detection in Underwater Sensor Networks by Fusion of Active and Passive Measurements

Target Detection by Decision-Level Fusion of Dual-Sensor Based on Evidence Theory

Research On Multi-Source Image Fusion Target Detection Technology Based on Neural Network

Target Detection with Distributed Terahertz Sensors

Multi-Sensors Data Tracking Fusion Based on a Neural Network Filter

Survey of Distributed Detection With Multisensor

Research on Spatial Target Recognition Method Based on Multi-source Sensor Fusion

An Asynchronous Data Fusion Algorithm for Target Detection Based on Multi-Sensor Networks

Distributed Source Number Estimation for Multiple Target Detection in Sensor Networks

Multi-Sensor Multi-Frame Detection Based on Posterior Probability Density Fusion.

Target Detection Using SAR/Optical Image Fusion Without Accurate Registration

Research on multi-target tracking method based on multi-sensor fusion

Target Tracking System for Multi-Sensor Data Fusion

Algorithm of Data Fusion Based on Neural Network Group

Adaptive Sensor Selection for Multitarget Detection in Heterogeneous Sensor Networks

Data Fusion of Multi-Target Tracking in Wireless Sensor Networks