Cyber Mobility Mirror: A Deep Learning-based Real-World Object Perception Platform Using Roadside LiDAR

Zhengwei Bai,Saswat Priyadarshi Nayak,Xuanpeng Zhao,Guoyuan Wu,Matthew J. Barth,Xuewei Qi,Yongkang Liu,Emrah Akin Sisbot,Kentaro Oguchi

DOI: https://doi.org/10.48550/arXiv.2202.13505

2022-04-08

Abstract:Object perception plays a fundamental role in Cooperative Driving Automation (CDA) which is regarded as a revolutionary promoter for the next-generation transportation systems. However, the vehicle-based perception may suffer from the limited sensing range and occlusion as well as low penetration rates in connectivity. In this paper, we propose Cyber Mobility Mirror (CMM), a next-generation real-time traffic surveillance system for 3D object perception and reconstruction, to explore the potential of roadside sensors for enabling CDA in the real world. The CMM system consists of six main components: 1) the data pre-processor to retrieve and preprocess the raw data; 2) the roadside 3D object detector to generate 3D detection results; 3) the multi-object tracker to identify detected objects; 4) the global locator to map positioning information from the LiDAR coordinate to geographic coordinate using coordinate transformation; 5) the cloud-based communicator to transmit perception information from roadside sensors to equipped vehicles, and 6) the onboard advisor to reconstruct and display the real-time traffic conditions via Graphical User Interface (GUI). In this study, a field-operational system is deployed at a real-world intersection, University Avenue and Iowa Avenue in Riverside, California to assess the feasibility and performance of our CMM system. Results from field tests demonstrate that our CMM prototype system can provide satisfactory perception performance with 96.99% precision and 83.62% recall. High-fidelity real-time traffic conditions (at the object level) can be geo-localized with an average error of 0.14m and displayed on the GUI of the equipped vehicle with a frequency of 3-4 Hz.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in Cooperative Driving Automation (CDA), vehicles may encounter limitations based on perception, such as limited perception range, occlusion, and low connection rate. To overcome these limitations, the author proposes a new real - time traffic monitoring system based on roadside LiDAR - Cyber Mobility Mirror (CMM) for 3D object detection and reconstruction, in order to explore the potential of roadside sensors to support CDA in the real world. Specifically, the CMM system aims to achieve real - time object - level traffic perception and reconstruction through high - precision sensors and advanced perception methods (such as object detection, classification, and tracking), thereby providing support for various cooperative driving automation applications, such as collision warning, Eco - Approach and Departure (EAD), Cooperative Adaptive Cruise Control (CACC), etc. The CMM system can generate high - fidelity 3D perception information, such as the geographical coordinates, 3D size, speed, and moving direction of detected objects. This information can be rendered and displayed on satellite maps, thereby constructing a "digital twin" environment as the perception basis for multiple CDA applications in the real - world traffic system.

Cyber Mobility Mirror: A Deep Learning-based Real-World Object Perception Platform Using Roadside LiDAR

Cyber Mobility Mirror for Enabling Cooperative Driving Automation in Mixed Traffic: A Co-Simulation Platform

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

Real-Time And Robust 3D Object Detection with Roadside LiDARs

Design, Implementation, and Evaluation of a Roadside Cooperative Perception System

Real-time Full-stack Traffic Scene Perception for Autonomous Driving with Roadside Cameras

Infrastructure-Based Object Detection and Tracking for Cooperative Driving Automation: A Survey

A Robust Strategy for Roadside Cooperative Perception Based on Multi-Sensor Fusion

InfraDet3D: Multi-Modal 3D Object Detection based on Roadside Infrastructure Camera and LiDAR Sensors

MSight: An Edge-Cloud Infrastructure-based Perception System for Connected Automated Vehicles

PillarGrid: Deep Learning-based Cooperative Perception for 3D Object Detection from Onboard-Roadside LiDAR

3D Environmental Perception Modeling in the Simulated Autonomous-Driving Systems

Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Clouds

Automatic Background Construction and Object Detection Based on Roadside LiDAR

A Cooperative Perception Environment for Traffic Operations and Control

Cooperative Perception with Deep Reinforcement Learning for Connected Vehicles

Center-Aware 3D Object Detection with Attention Mechanism Based on Roadside LiDAR

Influence of Camera-LiDAR Configuration on 3D Object Detection for Autonomous Driving

RMMDet: Road-Side Multitype and Multigroup Sensor Detection System for Autonomous Driving

Frontal Object Perception for Intelligent Vehicles Based on Radar and Camera Fusion

Real-Time 3D Object Detection and Classification in Autonomous Driving Environment Using 3D LiDAR and Camera Sensors