Abstract:The need for a vehicle to perceive information about the external environmental as an independent intelligent individual has grown with the progress of intelligent driving from primary driver assistance to high-level autonomous driving. The ability of a common independent sensing unit to sense the external environment is limited by the sensor’s own characteristics and algorithm level. Hence, a common independent sensing unit fails to obtain comprehensive sensing information independently under conditions such as rain, fog, and night. Accordingly, an extended network-based fusion target detection algorithm for millimeter-wave radar and vision fusion is proposed in this work by combining the complementary perceptual performance of in-vehicle sensing elements, cost effectiveness, and maturity of independent detection technologies. Feature-level fusion is first used in this work according to the analysis of technical routes of the millimeter-wave radar and vision fusion. Training and test evaluation of the algorithm are carried out on the nuScenes dataset and test data from a homemade data acquisition platform. An extended investigation on the RetinaNet one-stage target detection algorithm based on the VGG-16+FPN backbone detection network is then conducted in this work to introduce millimeter-wave radar images as auxiliary information for visual image target detection. We use two-channel radar and three-channel visual images as inputs of the fusion network. We also propose an extended VGG-16 network applicable to millimeter-wave radar and visual fusion and an extended feature pyramid network. Test results showed that the mAP of the proposed network improves by 2.9% and the small target accuracy is enhanced by 18.73% compared with those of the reference network for pure visual image target detection. This finding verified the detection capability and algorithmic feasibility of the proposed extended fusion target detection network for visually insensitive targets.

PAFNet: Pillar Attention Fusion Network for Vehicle–Infrastructure Cooperative Target Detection Using LiDAR

Fusing LiDAR and Radar with Pillars Attention for 3D Object Detection

Millimeter-Wave Radar and Vision Fusion Target Detection Algorithm Based on an Extended Network

FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving

VEHICLE-INFRASTRUCTURE COOPERATIVE 3D DETECTION VIA FEATURE FLOW PREDICTION

PA3DNet: 3-D Vehicle Detection with Pseudo Shape Segmentation and Adaptive Camera-LiDAR Fusion

AFRNet: Anchor-Free Object Detection Using Roadside LiDAR in Urban Scenes

Deep LiDAR-Radar-Visual Fusion for Object Detection in Urban Environments

ACF-Net: Asymmetric Cascade Fusion for 3D Detection with LiDAR Point Clouds and Images

RI-Fusion: 3D Object Detection Using Enhanced Point Features With Range-Image Fusion for Autonomous Driving.

AMFF-Net: An Effective 3D Object Detector Based on Attention and Multi-Scale Feature Fusion

VPFNet: Voxel-Pixel Fusion Network for Multi-class 3D Object Detection

Flow-Based Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detection

LiDAR-Camera Fusion: Guided Vehicle-to-Aircraft Sensing with Cooperative Vehicle Infrastructure

Vehicle-Infrastructure Cooperative 3D Object Detection via Feature Flow Prediction

Multi-View Adaptive Fusion Network for 3D Object Detection

BAFusion: Bidirectional Attention Fusion for 3D Object Detection Based on LiDAR and Camera

Timely Fusion of Surround Radar/Lidar for Object Detection in Autonomous Driving Systems

Multiattention Mechanism 3D Object Detection Algorithm Based on RGB and LiDAR Fusion for Intelligent Driving

Channelwise and Spatially Guided Multimodal Feature Fusion Network for 3-D Object Detection in Autonomous Vehicles

Adaptive and azimuth-aware fusion network of multimodal local features for 3D object detection