Abstract:Far-range perception through roadside sensors is crucial to the effectiveness of intelligent transportation systems. The main challenge of far-range perception is due to the difficulty of performing accurate object detection and tracking under far distances (e.g., > 150m) at a low cost. To cope with such challenges, deploying both millimeter wave Radars and high-definition (HD) cameras, and fusing their data for joint perception has become a common practice. The key to this solution, however, is the precise association between the two types of data, which are captured from different perspectives and have different degrees of measurement noises. Towards this goal, the first question is which plane to conduct the association, i.e., the 2D image plane or the BEV plane. We argue that the former is more suitable because the magnitude of location errors in the perspective projection points is smaller at far distances on the 2D plane and can lead to more accurate association. Thus, we first project the Radar-based target locations (on the BEV plane) to the 2D plane and then associate them with the camera-based object locations that are modeled as a point on each object. Subsequently, we map the camera-based object locations to the BEV plane through inverse projection mapping (IPM) with the corresponding depth information from the Radar data. Finally, we engage a BEV tracking module to generate target trajectories for traffic monitoring. Since our approach involves transformation between the 2D plane and BEV plane, we also devise a transformation parameters refining approach based on a depth scaling technique, utilising the above fusion process without requiring any additional devices such as GPS. We have deployed an actual testbed on an urban expressway and conducted extensive experiments to evaluate the effectiveness of our system. The results show that our system can improve AP BEV by 32%, and reduce the location error by 0.56m . Our system is capable of achieving an average location accuracy of 1.3m when we extend the detection range up to 500m . We thus believe that our proposed method offers a viable approach to efficient roadside far-range perception.

Real-Time Volumetric Perception for Unmanned Surface Vehicles Through Fusion of Radar and Camera

A Novel Unmanned Surface Vehicle with 2D-3D Fused Perception and Obstacle Avoidance Module

RaViDeep: Target Detection Based on Deep Fusion of Radar and Vision in Berthing Scenarios

Research on Unmanned Surface Vehicles Environment Perception Based on the Fusion of Vision and Lidar

Research on Surface Environment Perception Via Camera-LiDAR Sensor Fusion

Infrared and Visible Image Fusion Methods for Unmanned Surface Vessels with Marine Applications

Mask-VRDet: A Robust Riverway Panoptic Perception Model Based on Dual Graph Fusion of Vision and 4D Mmwave Radar

Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving

USV-Tracker: A Novel USV Tracking System for Surface Investigation with Limited Resources

Robust Small Object Detection on the Water Surface Through Fusion of Camera and Millimeter Wave Radar

UniBEVFusion: Unified Radar-Vision BEVFusion for 3D Object Detection

Obstacle Detection for Unmanned Surface Vehicles by Fusion Refinement Network

Vessel detection and classification fusing radar and vision data

Object Detection And Tracking Of Unmanned Surface Vehicles Based On Spatial-Temporal Information Fusion

A Multi-modality Sensor System for Unmanned Surface Vehicle

V2VFusion: Multimodal Fusion for Enhanced Vehicle-to-Vehicle Cooperative Perception

ASY-VRNet: Waterway Panoptic Driving Perception Model based on Asymmetric Fair Fusion of Vision and 4D mmWave Radar

FARFusion: A Practical Roadside Radar-Camera Fusion System for Far-Range Perception

Visual Temporal Fusion Based Free Space Segmentation for Autonomous Surface Vessels

A Fusion Algorithm of Object Detection and Tracking for Unmanned Surface Vehicles

A Millimeter-Wave Radar-Aided Vision Detection Method for Water Surface Small Object Detection