Abstract:Autonomous driving vehicles rely on sensors for the robust perception of their surroundings. Such vehicles are equipped with multiple perceptive sensors with a high level of redundancy to ensure safety and reliability in any driving condition. However, multi-sensor, such as camera, LiDAR, and radar systems raise requirements related to sensor calibration and synchronization, which are the fundamental blocks of any autonomous system. On the other hand, sensor fusion and integration have become important aspects of autonomous driving research and directly determine the efficiency and accuracy of advanced functions such as object detection and path planning. Classical model-based estimation and data-driven models are two mainstream approaches to achieving such integration. Most recent research is shifting to the latter, showing high robustness in real-world applications but requiring large quantities of data to be collected, synchronized, and properly categorized. However, there are two major research gaps in existing works: (i) they lack fusion (and synchronization) of multi-sensors, camera, LiDAR and radar; and (ii) generic scalable, and user-friendly end-to-end implementation. To generalize the implementation of the multi-sensor perceptive system, we introduce an end-to-end generic sensor dataset collection framework that includes both hardware deploying solutions and sensor fusion algorithms. The framework prototype integrates a diverse set of sensors, such as camera, LiDAR, and radar. Furthermore, we present a universal toolbox to calibrate and synchronize three types of sensors based on their characteristics. The framework also includes the fusion algorithms, which utilize the merits of three sensors, namely, camera, LiDAR, and radar, and fuse their sensory information in a manner that is helpful for object detection and tracking research. The generality of this framework makes it applicable in any robotic or autonomous applications and suitable for quick and large-scale practical deployment.

Datasets, Models, and Algorithms for Multi-Sensor, Multi-agent Autonomy Using AVstack

A Multi-Agent Security Testbed for the Analysis of Attacks and Defenses in Collaborative Sensor Fusion

A Modular Platform For Collaborative, Distributed Sensor Fusion

AVstack: An Open-Source, Reconfigurable Platform for Autonomous Vehicle Development

MSMA: Multi-agent Trajectory Prediction in Connected and Autonomous Vehicle Environment with Multi-source Data Integration

End-to-End Multimodal Sensor Dataset Collection Framework for Autonomous Vehicles

Multi-agent collaboration environment simulation

Robustness and Adaptability of Reinforcement Learning-Based Cooperative Autonomous Driving in Mixed-Autonomy Traffic

A FRAMEWORK FOR MULTI-SENSOR POSITIONING AND MAPPING FOR AUTONOMOUS VEHICLES

Cooperation for Scalable Supervision of Autonomy in Mixed Traffic

Multi-agent reinforcement learning for autonomous vehicles: a survey

Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset

Multi-Agent Probabilistic Ensembles with Trajectory Sampling for Connected Autonomous Vehicles

Social Coordination and Altruism in Autonomous Driving

A Multimodal Perception-Driven Self Evolving Autonomous Ground Vehicle

Multi-robot, multi-sensor exploration of multifarious environments with full mission aerial autonomy

Autonomous Advanced Aerial Mobility -- An End-to-end Autonomy Framework for UAVs and Beyond

Towards Sensor Data Abstraction of Autonomous Vehicle Perception Systems

High-Resolution Traffic Sensing with Autonomous Vehicles

AVA: Towards Autonomous Visualization Agents through Visual Perception-Driven Decision-Making

Dataset and Benchmark: Novel Sensors for Autonomous Vehicle Perception