Abstract:Object detection is the central issue of intelligent traffic systems, and recent advancements in single-vehicle lidar-based 3D detection indicate that it can provide accurate position information for intelligent agents to make decisions and plan. Compared with single-vehicle perception, multi-view vehicle-road cooperation perception has fundamental advantages, such as the elimination of blind spots and a broader range of perception, and has become a research hotspot. However, the current perception of cooperation focuses on improving the complexity of fusion while ignoring the fundamental problems caused by the absence of single-view outlines. We propose a multi-view vehicle-road cooperation perception system, vehicle-to-everything cooperative perception (V2X-AHD), in order to enhance the identification capability, particularly for predicting the vehicle's shape. At first, we propose an asymmetric heterogeneous distillation network fed with different training data to improve the accuracy of contour recognition, with multi-view teacher features transferring to single-view student features. While the point cloud data are sparse, we propose Spara Pillar, a spare convolutional-based plug-in feature extraction backbone, to reduce the number of parameters and improve and enhance feature extraction capabilities. Moreover, we leverage the multi-head self-attention (MSA) to fuse the single-view feature, and the lightweight design makes the fusion feature a smooth expression. The results of applying our algorithm to the massive open dataset V2Xset demonstrate that our method achieves the state-of-the-art result. The V2X-AHD can effectively improve the accuracy of 3D object detection and reduce the number of network parameters, according to this study, which serves as a benchmark for cooperative perception. The code for this article is available at https://github.com/feeling0414-lab/V2X-AHD.

V2X-ViTv2: Improved Vision Transformers for Vehicle-to-Everything Cooperative Perception

V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer

HM-ViT: Hetero-modal Vehicle-to-Vehicle Cooperative perception with vision transformer

V2X-AHD:Vehicle-to-Everything Cooperation Perception via Asymmetric Heterogenous Distillation Network

Dynamic V2X Autonomous Perception from Road-to-Vehicle Vision

V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction

S2R-ViT for Multi-Agent Cooperative Perception: Bridging the Gap from Simulation to Reality

V2X Cooperative Perception for Autonomous Driving: Recent Advances and Challenges

V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving

OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication

CoFormerNet: A Transformer-Based Fusion Approach for Enhanced Vehicle-Infrastructure Cooperative Perception

Multi-V2X: A Large Scale Multi-modal Multi-penetration-rate Dataset for Cooperative Perception

CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers

V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception

DI-V2X: Learning Domain-Invariant Representation for Vehicle-Infrastructure Collaborative 3D Object Detection

A Novel Probabilistic V2X Data Fusion Framework for Cooperative Perception

Consensus-Based Distributed Cooperative Perception for Connected and Automated Vehicles

NLOS Dies Twice: Challenges and Solutions of V2X for Cooperative Perception

V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Models