Abstract:Abstract The task of environmental perception is an extremely important part of the automatic driving project, and mainstream schemes are divided into visual perception and radar perception. Visual algorithms have advantages in cost and object detection or image segmentation methods are usually used for perception. We adopt computer vision scheme to collect data and establish Transportation of Wuhan China (TOWC) datasets based on Chinese urban situation. After optimizing the selection of anchor boxes, the Yolov4 model is retrained as the object detector, which is combined with Simple Online and Real-time Tracking (SORT) algorithm and Perspective-n-Point (PnP) monocular algorithm to realize the integration of object detection, tracking and ranging. The detection and tracking methods have been evaluated against the Microsoft Common Objects in Context (MSCOCO) datasets and Multiple Object Tracking (MOT) challenge respectively. And the detection performance also has been evaluated again in our TOWC datasets. The evaluation has been conducted in challenging conditions, including occlusion, partial visibility, and under lighting variations with the mean average precision of 97.17% and the real-time speed of 30.2 FPS. The results show that our fusion detection algorithm is not inferior to other most advanced methods on the premise of excellent real-time performance. The developed model is a generic and accurate solution for traffic object detection, tracking and ranging, which can be applied to many other fields, such as automatic driving vehicle, traffic sign recognition, anomaly detection, motion analysis, or any other research areas where the traffic environmental perception is in the center of attention.

Multimedia Fusion at Semantic Level in Vehicle Cooperactive Perception

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

Semantic Fusion Infrastructure for Unmanned Vehicle System Based on Cooperative 5G MEC

Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving

Occlusion-Guided Multi-Modal Fusion for Vehicle-Infrastructure Cooperative 3D Object Detection

Enhancing 3D object detection through multi-modal fusion for cooperative perception

Cooperative Visual Augmentation Algorithm of Intelligent Vehicle Based on Inter-Vehicle Image Fusion

Semantic Communication for Cooperative Perception based on Importance Map

Human-vehicle Cooperative Visual Perception for Autonomous Driving under Complex Road and Traffic Scenarios

Multi-sensor fusion algorithm in cooperative vehicle-infrastructure system for blind spot warning

Cooperative Perception With Learning-Based V2V Communications

Autonomous Multi-Sensor Fusion Techniques for Environmental Perception in Self-Driving Vehicles

Cooperative Perception Technology of Autonomous Driving in the Internet of Vehicles Environment: A Review

SmartCooper: Vehicular Collaborative Perception with Adaptive Fusion and Judger Mechanism

Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Clouds

Multi-modal Sensor Fusion for Auto Driving Perception: A Survey

Multi-Sensor Fusion in Automated Driving: A Survey

EMIFF: Enhanced Multi-scale Image Feature Fusion for Vehicle-Infrastructure Cooperative 3D Object Detection

Enhanced Perception for Autonomous Driving Using Semantic and Geometric Data Fusion

ERMVP: Communication-Efficient and Collaboration-Robust Multi-Vehicle Perception in Challenging Environments