Abstract:Sharing and joint processing of camera feeds and sensor measurements, known as Cooperative Perception (CP), has emerged as a new technique to achieve higher perception qualities. CP can enhance the safety of Autonomous Vehicles (AVs) where their individual visual perception quality is compromised by adverse weather conditions (haze as foggy weather), low illumination, winding roads, and crowded traffic. To cover the limitations of former methods, in this paper, we propose a novel approach to realize an optimized CP under constrained communications. At the core of our approach is recruiting the best helper from the available list of front vehicles to augment the visual range and enhance the Object Detection (OD) accuracy of the ego vehicle. In this two-step process, we first select the helper vehicles that contribute the most to CP based on their visual range and lowest motion blur. Next, we implement a radio block optimization among the candidate vehicles to further improve communication efficiency. We specifically focus on pedestrian detection as an exemplary scenario. To validate our approach, we used the CARLA simulator to create a dataset of annotated videos for different driving scenarios where pedestrian detection is challenging for an AV with compromised vision. Our results demonstrate the efficacy of our two-step optimization process in improving the overall performance of cooperative perception in challenging scenarios, substantially improving driving safety under adverse conditions. Finally, we note that the networking assumptions are adopted from LTE Release 14 Mode 4 side-link communication, commonly used for Vehicle-to-Vehicle (V2V) communication. Nonetheless, our method is flexible and applicable to arbitrary V2V communications.

Decentralized Multiagent Reinforcement Learning-based Cooperative Perception with Dual-functional Radar-Communication V2V Links

Delay-Aware Cooperative Perception with Deep Reinforcement Learning in Vehicular Networks

Model-Assisted Learning for Adaptive Cooperative Perception of Connected Autonomous Vehicles

PACP: Priority-Aware Collaborative Perception for Connected and Autonomous Vehicles

Cooperative Sensing and Heterogeneous Information Fusion in VCPS: A Multi-agent Deep Reinforcement Learning Approach

Cooperative Perception with Deep Reinforcement Learning for Connected Vehicles

Cooperative Localisation for Multi-Rsu Vehicular Networks Based on Predictive Beamforming

V2I Interference Management for Collaborative Perception Based on Hierarchical Reinforcement Learning

Enhancing Integrated Sensing and Communication (ISAC) Performance for a Searching–Deciding Alternation Radar-Comm System with Multi-Dimension Point Cloud Data

RSU-Aided Energy-Efficient Collaborative Perception for Connected Autonomous Vehicles

Learning to Schedule Joint Radar-Communication Requests for Optimal Information Freshness

Reinforcement Learning-Based Resource Allocation for Multiple Vehicles with Communication-Assisted Sensing Mechanism

Communication-Efficient Decentralized Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control

Decentralized cooperative perception for autonomous vehicles: Learning to value the unknown

Enhanced Automotive Sensing Assisted by Joint Communication and Cognitive Sparse MIMO Radar

Reinforcement Learning Based Integrated Sensing and Communication for Automotive MIMO Radar

Soft Actor–Critic-Based Multilevel Cooperative Perception for Connected Autonomous Vehicles

InterCoop: Spatio-Temporal Interaction Aware Cooperative Perception for Networked Vehicles

Enhanced Cooperative Perception for Autonomous Vehicles Using Imperfect Communication

MACP: Efficient Model Adaptation for Cooperative Perception