Abstract:The value of roadside perception, which could extend the boundaries of autonomous driving and traffic management, has gradually become more prominent and acknowledged in recent years. However, existing roadside perception approaches only focus on the single-infrastructure sensor system, which cannot realize a comprehensive understanding of a traffic area because of the limited sensing range and blind spots. Orienting high-quality roadside perception, we need Roadside Cooperative Perception (RCooper) to achieve practical area-coverage roadside perception for restricted traffic areas. Rcooper has its own domain-specific challenges, but further exploration is hindered due to the lack of datasets. We hence release the first real-world, large-scale RCooper dataset to bloom the research on practical roadside cooperative perception, including detection and tracking. The manually annotated dataset comprises 50k images and 30k point clouds, including two representative traffic scenes (i.e., intersection and corridor). The constructed benchmarks prove the effectiveness of roadside cooperation perception and demonstrate the direction of further research. Codes and dataset can be accessed at:

What problem does this paper attempt to address?

This paper proposes a solution to the problem of Roadside Cooperative Perception (RCooper). With the development of the Internet of Things, 5G, and artificial intelligence technologies, the value of roadside perception in autonomous driving and traffic management is becoming more prominent. However, existing roadside perception methods mainly rely on sensor systems of single infrastructure, which leads to limited perception range and blind spots, and fails to achieve a comprehensive understanding of the traffic area. In order to achieve high-quality roadside perception, the paper introduces the concept of RCooper, aiming to enhance autonomous driving and traffic management through cross-infrastructure cooperation to cover restricted traffic areas. The challenges of existing methods include data heterogeneity, the need to improve cooperation representation, and the improvement of perception performance. The lack of relevant datasets currently hinders research in this field. Therefore, the paper releases the first real-world, large-scale RCooper dataset, which includes 50,000 images and 30,000 point cloud data, covering two representative traffic scenarios (intersection and corridor). These data are manually annotated and used for detection and tracking tasks, demonstrating the effectiveness of roadside cooperative perception and pointing out future research directions. In addition, the paper compares existing roadside perception, cooperative perception, and public perception datasets, and introduces data collection methods, coordinate and data annotation, and data statistical analysis. The experimental results show that cooperative perception performs better than non-cooperative methods in corridor scenarios, but in intersection scenarios, late fusion strategies perform better in certain indicators, but also reveal the challenges brought by data heterogeneity. Overall, this paper focuses on addressing how to extend the perception range, reduce blind spots, and improve the efficiency of autonomous driving and traffic management through roadside cooperative perception, and provides new datasets and benchmarks for this purpose.

RCooper: A Real-world Large-scale Dataset for Roadside Cooperative Perception

CORP: A Multi-Modal Dataset for Campus-Oriented Roadside Perception Tasks

V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception

Rope3D: TheRoadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task

TUMTraf V2X Cooperative Perception Dataset

CoPeD-Advancing Multi-Robot Collaborative Perception: A Comprehensive Dataset in Real-World Environments

A Robust Strategy for Roadside Cooperative Perception Based on Multi-Sensor Fusion

RoScenes: A Large-scale Multi-view 3D Dataset for Roadside Perception

V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception

Design, Implementation, and Evaluation of a Roadside Cooperative Perception System

V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception

Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Clouds

ROCO: A Roundabout Traffic Conflict Dataset

RSRD: A Road Surface Reconstruction Dataset and Benchmark for Safe and Comfortable Autonomous Driving

EdgeCooper: Network-Aware Cooperative LiDAR Perception for Enhanced Vehicular Awareness

Multi-V2X: A Large Scale Multi-modal Multi-penetration-rate Dataset for Cooperative Perception

DOLPHINS: Dataset for Collaborative Perception enabled Harmonious and Interconnected Self-driving

Optimization of Roadside Sensors Placement for Cooperative Vehicle-Infrastructure System

RoCo: Robust Cooperative Perception by Iterative Object Matching and Pose Adjustment

RoboSense: Large-scale Dataset and Benchmark for Egocentric Robot Perception and Navigation in Crowded and Unstructured Environments

Scalability in Perception for Autonomous Driving: Waymo Open Dataset.